-- Logs begin at Mon 2022-08-29 20:27:24 UTC, end at Mon 2022-08-29 21:51:55 UTC. -- Aug 29 21:02:52.137200 dv-cinder-854264-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 21:02:52.148298 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 21:02:52 2022] *** Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: nodename: dv-cinder-854264-2-iscsi Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: machine: x86_64 Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: clock source: unix Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: pcre jit disabled Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: detected number of CPU cores: 4 Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: current working directory: / Aug 29 21:02:52.150341 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: detected binary path: /usr/bin/uwsgi-core Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: your processes number limit is 47680 Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: your memory page size is 4096 bytes Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: detected max file descriptor number: 2048 Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: lock engine: pthread robust mutexes Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: thunder lock: enabled Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 21:02:52.150985 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 21:02:52.190207 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: Python main interpreter initialized at 0x5557c109f9c0 Aug 29 21:02:52.190207 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: python threads support enabled Aug 29 21:02:52.190207 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: your server socket listen backlog is limited to 100 connections Aug 29 21:02:52.190207 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: your mercy for graceful operations on workers is 90 seconds Aug 29 21:02:52.190602 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 21:02:52.190670 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: *** Operational MODE: preforking *** Aug 29 21:02:52.191583 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: *** uWSGI is running in multiple interpreter mode *** Aug 29 21:02:52.191583 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: spawned uWSGI master process (pid: 81863) Aug 29 21:02:52.191798 dv-cinder-854264-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 21:02:52.193648 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: spawned uWSGI worker 1 (pid: 81874, cores: 1) Aug 29 21:02:52.193648 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: spawned uWSGI worker 2 (pid: 81875, cores: 1) Aug 29 21:02:52.193648 dv-cinder-854264-2-iscsi devstack@keystone.service[81863]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 21:02:56.638748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:02:56.638748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: from cryptography.utils import int_from_bytes Aug 29 21:02:56.639562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:02:56.639562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: from cryptography.utils import int_from_bytes Aug 29 21:02:56.639674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:02:56.639674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: from cryptography.utils import int_from_bytes Aug 29 21:02:56.640129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 21:02:56.640129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: from cryptography.utils import int_from_bytes Aug 29 21:02:56.689910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 21:02:56.690090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 21:02:56.690185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 21:02:56.690267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 21:02:56.690339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] command line args: [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 21:02:56.690447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] command line args: [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 21:02:56.690592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 21:02:56.690724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 21:02:56.690827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ================================================================================ {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 21:02:56.690912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ================================================================================ {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 21:02:56.690989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] admin_token = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] admin_token = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] config_dir = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] config_dir = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] config_source = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] debug = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] config_source = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.691982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] debug = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.692071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.693962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_config_append = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_config_append = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_dir = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_dir = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_file = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_file = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.694985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.695969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.696918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] publish_errors = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.697985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] publish_errors = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.698953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] standard_threads = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] standard_threads = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.699935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] use_journal = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] use_journal = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] use_json = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] use_json = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] use_stderr = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] use_stderr = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] use_syslog = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] use_syslog = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.700907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 21:02:56.701022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.701848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.702986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.703993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.704950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.705936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.706583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.706705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.706785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.706969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.707953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708422 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.708922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.709950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.710110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.710933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.710933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.711958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.712926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.713932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.714941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.715964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.716934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.717983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.718945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.719932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.720898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.721046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.721138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.721282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.721370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.721517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.721616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.722320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.722417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.722590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.722705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.722827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.722955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.723866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.724914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.725992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726062 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726422 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.726916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.external = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.external = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.727942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.password = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.password = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] auth.token = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.728948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] auth.token = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.729923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.730945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.731966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.732985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.733978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.734946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.735929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.736920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.737911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.738958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.739825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.740951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.741950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.742951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.743879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744062 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.744883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.745846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.746937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.747878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.748882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.749993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.750939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.751922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.752710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.753391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.753477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.753628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.753711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.753858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.753938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.754912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.755866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.756980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.757931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.758879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.759822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.760843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.761979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] role.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] role.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] role.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] role.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.762930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.763944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764444 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.764930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765008 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.765903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.766875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.767872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.768058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.768176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.768313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.768394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.769947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.770862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.771966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.772999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.773995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.774924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.775967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.connection = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.connection = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.776931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.777877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.778981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.779962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.780056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.780193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.780272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.780367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81874) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 21:02:56.780447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.780659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 21:02:56.780829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81875) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 21:02:56.810016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.810211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.810284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.810363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.810437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.810536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.819463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.819623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.848476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.848700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.848822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.848935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.903970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.904141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.954407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.954644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.954733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:56.954811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.114807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.115957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.116042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.116188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.116411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.116630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.139527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.139689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.139776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.139854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.140950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.152590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.152868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.166866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81875) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.167648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81874) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 21:02:57.184433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.184644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:02:57.184809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.184991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.185075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:02:57.185192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:02:57.185358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.185451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.185562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 21:02:57.185717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.185847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.185979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.186113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.186237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.186642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.186911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.196119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.196391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.196501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.196660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.196744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.196943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.197024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.197230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.203908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.204216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.207307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.207593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.207673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.207900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.207985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.208190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.208271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.208477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.208570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.208744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.208825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.209949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.210104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.210235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.210513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.232862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.233124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.234323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.234599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.234681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.234888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.239862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.240171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.240367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.240449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.240658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.240946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.246857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.247146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.247283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.247575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.251062 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.251359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.251655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.251738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.251980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.252063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.252259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.252339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.252547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.253379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.253705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.254006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.254297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.254917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.255211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.255512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.271116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.271436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.271617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.272115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.275326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.275776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.278399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.278708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.279037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.279340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.283004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.283560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.286176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.286771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.289344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.289652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.289942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.290967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.291255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.291484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.307079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.307372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.307642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.307825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.307926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.308147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.308425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.308691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.314602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.316392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.317002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.318027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.318539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.318818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.319588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.319872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.321456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.321748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.322042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.322330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.322536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.322616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.322840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.323249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.323538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.323839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.334099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.334439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.334725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.335008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.335119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.335292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.335412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.335572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.335800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.335901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.336118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.336411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.336700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.337003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.351144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.352397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.354184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.355298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.357054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.358194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.358499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.359627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.359992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.360276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.360571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.361269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.361559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.361861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.367999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.369151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.369482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.370640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.372361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.372664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.372949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.373235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.373664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.373961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.374251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.374542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.381652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.381946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.382235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.382950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.383246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.383531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.389181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81875) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.389468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.389751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.390033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.390311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.390460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81874) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 21:02:57.390607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.390750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.390883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.391915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.392847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81875) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.393042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.393314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.393615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.393885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.394164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81874) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 21:02:57.427263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5557c109f9c0 pid: 81875 (default app) Aug 29 21:02:57.428711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5557c109f9c0 pid: 81874 (default app) Aug 29 21:02:57.430821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f578eb0-b101-4f41-8be4-218f81062082 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:02:57.431105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f578eb0-b101-4f41-8be4-218f81062082 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:02:57.431378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f578eb0-b101-4f41-8be4-218f81062082 None None] PATH_INFO: `/v3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:02:57.435487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Mon Aug 29 21:02:57 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:05.290644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:05.291216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:05.291216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:05.295449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:05.301218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:05.301792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:05.301792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:05.372341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: INFO dbcounter [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] Registered counter for database keystone Aug 29 21:03:05.394577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG oslo_db.sqlalchemy.engines [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81875) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 21:03:05.401013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writer thread running {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 21:03:05.582181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG passlib.handlers.bcrypt [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81875) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 21:03:05.582569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG passlib.handlers.bcrypt [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81875) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 21:03:05.641825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:05.645384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:05.776925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 21:03:06.260734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:06.330301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: INFO dbcounter [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] Registered counter for database keystone Aug 29 21:03:06.348998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG oslo_db.sqlalchemy.engines [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81874) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 21:03:06.355580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writer thread running {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 21:03:06.475226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61abcd3-076a-48ee-95ab-36f2397aa554 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:06.479553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:06.480217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:06.480499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:06.480778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:06.489843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:06.542058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 21:03:06.542403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5934b0-c62c-4df8-970b-b483a48e3982 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:06.553036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Mon Aug 29 21:03:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 21:03:06.559126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:06.573684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c34c9-78e8-4bec-83fa-7a9277ec1436 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:06.578192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:06.579294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:06.579580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:06.579926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] PATH_INFO: `/v3/projects/eb1de965aba54ca98b770d7165c4b67e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:06.581575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] RBAC: Authorizing `identity:get_project(project_id=eb1de965aba54ca98b770d7165c4b67e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:06.634979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING keystone.common.rbac_enforcer.enforcer [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 21:03:06.635330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06628a4e-42e6-493c-b8cf-d4ae78e94d56 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:06.640124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Mon Aug 29 21:03:06 2022] GET /identity/v3/projects/eb1de965aba54ca98b770d7165c4b67e => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:08.528187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ac748-e3b0-4216-9831-042687655fc4 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:08.528700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ac748-e3b0-4216-9831-042687655fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:08.528700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ac748-e3b0-4216-9831-042687655fc4 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:08.532809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:08.538840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:08.539251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:08.539397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:08.561317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:08.562890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:08.571126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 21:03:08.840883 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ac748-e3b0-4216-9831-042687655fc4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:08.854582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ac748-e3b0-4216-9831-042687655fc4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:08.858822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65002b63-c2a7-4498-a625-080ba54053db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:08.859435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65002b63-c2a7-4498-a625-080ba54053db admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:08.859720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65002b63-c2a7-4498-a625-080ba54053db admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:08.860039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65002b63-c2a7-4498-a625-080ba54053db admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:08.868335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65002b63-c2a7-4498-a625-080ba54053db admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:08.870169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65002b63-c2a7-4498-a625-080ba54053db admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:08.879574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Mon Aug 29 21:03:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:03:08.885614 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:08.895803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3050c-85b4-4e84-9485-4c2d7ed1a83b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:08.900287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489f8cb-bf32-4279-8635-f541c21859d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:08.900933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f8cb-bf32-4279-8635-f541c21859d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:08.901218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f8cb-bf32-4279-8635-f541c21859d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:08.901496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f8cb-bf32-4279-8635-f541c21859d6 admin admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:08.902836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489f8cb-bf32-4279-8635-f541c21859d6 admin admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:08.904753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489f8cb-bf32-4279-8635-f541c21859d6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:08.906350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Mon Aug 29 21:03:08 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:10.911117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147386d-02b9-428d-b0ef-816b1e27a206 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:10.911665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147386d-02b9-428d-b0ef-816b1e27a206 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:10.911808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147386d-02b9-428d-b0ef-816b1e27a206 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:10.913180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:10.918391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:10.918823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:10.919002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:10.933633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe131eb-84e9-4745-9b20-a21be2979254 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:10.933916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe131eb-84e9-4745-9b20-a21be2979254 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:10.934195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe131eb-84e9-4745-9b20-a21be2979254 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:10.935510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 6/11] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:10.937423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:10.938609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:10.940619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:10.940906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:10.941188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:10.984178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG passlib.handlers.bcrypt [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81874) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 21:03:10.986419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG passlib.handlers.bcrypt [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81874) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 21:03:11.022516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 6/12] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:11.036293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:11.037947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:11.143467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 7/13] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:11.291099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.319654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:11.405531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9212ba2-f392-4e7f-b1e0-f9d37501c577 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:11.411798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.439372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:11.440050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:11.440338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:11.440618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:11.452000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:11.461158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:11.462197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:11.463073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 7/14] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:11.472287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.513210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd99c43-192f-4d4d-bb99-e1eb4aa14694 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:11.515582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:11.559287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:11.560114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:11.560568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:11.561068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:11.567395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:11.586310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3a6f-1f77-4d47-a690-ff97bc48b8cd None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:11.623495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:11.624151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:11.624440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:11.624717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:11.625393 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:11.625699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:11.627150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:11.636862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:11.638401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:11.639173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 8/15] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:11.645411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.747823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 8/16] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:03:11.753872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.754959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1f32b-f08c-4b1c-b249-f2c03be1339f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:11.757617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:11.789875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:11.790516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:11.790799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:11.791075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:11.792283 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:11.793788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:11.795395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 9/17] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:11.801021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.815854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:11.855470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ad7f7-5b85-46b7-a4f6-f162d568d0e8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:11.896236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:11.897916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:11.898359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:11.898772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:11.916525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b8b29-ba2f-492c-83d3-a6aa08d16f09 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:11.920661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:11.921666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:03:11.942475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:11.943395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:03:11.944266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:03:11.944266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:03:11.944266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:11.944266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 9/18] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:11.950189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:11.963557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:11.964432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:11.964780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:11.965060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:11.966268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:11.967939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:11.971241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 10/19] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:11.977215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:12.024123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:12.075985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:12.085796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536a918-c936-4b8e-94b2-3ddace68243d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:12.098927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e99ea-2f02-4c86-a8ec-30b6360e6ed5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:12.132671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:12.133390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:12.133679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:12.133956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:12.134741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:12.135177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:12.136770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e2350-60ae-4e6d-bb63-3a08a945b6ce None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:12.145155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:12.145780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:12.146064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:12.146343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:12.147172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.domain.id=default, user.id=6fd5eb0c7a1f4bdbb078438d56380575) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:12.147550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:12.149159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c020a-9b4c-4642-ab06-f361aae922aa None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:12.156294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 10/20] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:12.173743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 11/21] 10.222.0.65 () {56 vars in 1322 bytes} [Mon Aug 29 21:03:11 2022] GET /identity/v3/role_assignments?user.id=6fd5eb0c7a1f4bdbb078438d56380575&scope.domain.id=default&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 233 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:14.323254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a3430-892f-4c23-bd09-0759a1e63506 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:14.324262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a3430-892f-4c23-bd09-0759a1e63506 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:14.324769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a3430-892f-4c23-bd09-0759a1e63506 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:14.326317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 11/22] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:14.332108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe16c6d-31ac-4e0b-a17a-1ffbc0b3c0e3 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:14.332951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe16c6d-31ac-4e0b-a17a-1ffbc0b3c0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:14.333434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe16c6d-31ac-4e0b-a17a-1ffbc0b3c0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:14.361131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ffe16c6d-31ac-4e0b-a17a-1ffbc0b3c0e3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:14.363807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ffe16c6d-31ac-4e0b-a17a-1ffbc0b3c0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:14.449296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 12/23] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:14.638642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec9319-41cb-4052-a939-33a1c4577d47 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:14.639318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec9319-41cb-4052-a939-33a1c4577d47 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:14.639801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec9319-41cb-4052-a939-33a1c4577d47 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:14.641435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 12/24] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:14.647966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:14.649237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:14.649712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:14.676995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:14.678562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:14.748699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d706d-afe7-475e-891c-e6d905c77447 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:14.749309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d706d-afe7-475e-891c-e6d905c77447 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:14.749780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d706d-afe7-475e-891c-e6d905c77447 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:14.751953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 13/25] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:14.765611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:14.766208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:14.766746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:14.799738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:14.802532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:14.809833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 13/26] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:14.902073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 14/27] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:14.937262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:14.941720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:15.058495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354fb74-8203-4a2c-b681-a97cdedabdb1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:15.111902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28032899-b321-4320-b212-a2a277a562fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:15.112804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28032899-b321-4320-b212-a2a277a562fa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.113080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28032899-b321-4320-b212-a2a277a562fa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.113359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28032899-b321-4320-b212-a2a277a562fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.136862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28032899-b321-4320-b212-a2a277a562fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-28032899-b321-4320-b212-a2a277a562fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:15.142459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:15.144470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:15.146005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:15.156452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:15.217167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:15.281803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3257b-e581-4f72-8769-fefe8760070c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:15.328249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:15.329252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.329252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.329522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.330238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:15.330546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:15.332149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:15.341940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df047a3-4203-407b-b85a-1a376d7da4b8 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.342647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df047a3-4203-407b-b85a-1a376d7da4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.342647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df047a3-4203-407b-b85a-1a376d7da4b8 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.344190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 15/29] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:15.349456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.349812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.350075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.357462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 15/30] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:15.367361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:15.385460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:15.386680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:15.494368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:15.505375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 16/31] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:15.506443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:15.626398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af7b3-c0bd-4e1f-a8a7-596d07add960 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:15.628560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c331-519c-4886-843a-fa8f397d8b33 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:15.664390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:15.672937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:15.673774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.679185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.679185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.679185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:15.679185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:15.707600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:15.708758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.709185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.709637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.711372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:15.713191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:15.715035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 16/32] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:15 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:15.715972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:15.846022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405da5ac-4dbd-4f74-8ab2-31fbdc17b936 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:15.885110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:15.886024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:15.886474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:15.886894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:15.899262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:15.901017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:03:15.918020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:15.918997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:03:15.919800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:03:15.919800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:03:15.919800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:15.921604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 17/33] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:15.922622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:15.958011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:15.994167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 17/34] 10.222.0.65 () {60 vars in 1131 bytes} [Mon Aug 29 21:03:15 2022] POST /identity/v3/roles => generated 219 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:15.996732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:16.044073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ba335-0bac-4da2-a6f4-2b970bb11e5a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:16.086140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:16.087036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.087493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.088043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.089418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:16.091132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:16.093183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 18/35] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:16.094275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:16.122625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:16.217095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:16.217346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566707-e95d-434d-b7a8-2ca429dffa6b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:16.224978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d061-e81c-4489-8ddc-96e55fd2b188 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:16.265264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:16.266328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.266817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.267295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.268236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:16.268765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:16.270584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:16.284242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 19/36] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:16.285433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:16.297905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:16.299421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.300136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.300934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.302332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:16.305646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:16.380043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 18/37] 10.222.0.65 () {60 vars in 1131 bytes} [Mon Aug 29 21:03:15 2022] POST /identity/v3/roles => generated 225 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:16.382157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:16.408564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a69c-a70a-4b17-ac7f-829585aafdc0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:16.452276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd27576-d830-4e75-83fc-002255a9bcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:16.453087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd27576-d830-4e75-83fc-002255a9bcb3 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.453380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd27576-d830-4e75-83fc-002255a9bcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.453710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd27576-d830-4e75-83fc-002255a9bcb3 None admin] PATH_INFO: `/v3/domains/default/users/6fd5eb0c7a1f4bdbb078438d56380575/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.456507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd27576-d830-4e75-83fc-002255a9bcb3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6fd5eb0c7a1f4bdbb078438d56380575, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:16.458074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd27576-d830-4e75-83fc-002255a9bcb3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:16.555125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2a86b-186a-48dd-a01e-e6f7477e7c71 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:16.571472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:16.597069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b651298-e9d9-42e4-9f3d-ae2069dcf713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:16.597836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b651298-e9d9-42e4-9f3d-ae2069dcf713 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.598194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b651298-e9d9-42e4-9f3d-ae2069dcf713 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.598506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b651298-e9d9-42e4-9f3d-ae2069dcf713 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.599244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b651298-e9d9-42e4-9f3d-ae2069dcf713 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:16.600900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b651298-e9d9-42e4-9f3d-ae2069dcf713 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:16.715917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 19/38] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:03:16 2022] POST /identity/v3/projects => generated 301 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:16.715917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10414d-bc2d-4861-87e8-004663442430 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.715917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10414d-bc2d-4861-87e8-004663442430 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.715917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10414d-bc2d-4861-87e8-004663442430 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.716563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 20/39] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:16.722797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70e849-ff2d-4412-9f6f-e1c9fb6d57db None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:16.723090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70e849-ff2d-4412-9f6f-e1c9fb6d57db None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:16.723315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70e849-ff2d-4412-9f6f-e1c9fb6d57db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:16.743439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 20/40] 10.222.0.65 () {58 vars in 1361 bytes} [Mon Aug 29 21:03:16 2022] PUT /identity/v3/domains/default/users/6fd5eb0c7a1f4bdbb078438d56380575/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:03:16.747917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-dd70e849-ff2d-4412-9f6f-e1c9fb6d57db None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:16.750152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-dd70e849-ff2d-4412-9f6f-e1c9fb6d57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:16.850272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 21/41] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:17.057226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b342c-b4eb-40df-ac58-ddb37883399d None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:17.057498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b342c-b4eb-40df-ac58-ddb37883399d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:17.057784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b342c-b4eb-40df-ac58-ddb37883399d None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:17.063011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 21/42] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:17.068642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:17.068933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:17.069219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:17.088981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:17.090185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:17.133961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b342c-b4eb-40df-ac58-ddb37883399d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:17.154735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:17.268204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b342c-b4eb-40df-ac58-ddb37883399d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:17.277701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 22/43] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:17.308280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:17.308914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:17.309204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:17.309537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:17.310185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:17.311806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:17.423144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 22/44] 10.222.0.65 () {60 vars in 1131 bytes} [Mon Aug 29 21:03:17 2022] POST /identity/v3/roles => generated 223 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:03:17.548412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:17.579239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:17.670694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1680cf-f2d3-4262-9f17-882b7802a48f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:17.709439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3a2c1-7ee5-4003-87b4-dd4598e2d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:17.709439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a2c1-7ee5-4003-87b4-dd4598e2d4fd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:17.709439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a2c1-7ee5-4003-87b4-dd4598e2d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:17.709439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a2c1-7ee5-4003-87b4-dd4598e2d4fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:17.710604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3a2c1-7ee5-4003-87b4-dd4598e2d4fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:17.712169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc3a2c1-7ee5-4003-87b4-dd4598e2d4fd None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:17.713785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 23/45] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:17.720013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:17.799195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:17.833213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be9ca6-1545-4a73-bdc4-77392056d9aa None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:17.871559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317e2a-fc3e-4ae1-86b6-25ad22509f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:17.872265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317e2a-fc3e-4ae1-86b6-25ad22509f91 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:17.872571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317e2a-fc3e-4ae1-86b6-25ad22509f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:17.872852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317e2a-fc3e-4ae1-86b6-25ad22509f91 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:17.873530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317e2a-fc3e-4ae1-86b6-25ad22509f91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:17.875014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317e2a-fc3e-4ae1-86b6-25ad22509f91 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:17.944045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 23/46] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:03:17 2022] POST /identity/v3/projects => generated 312 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:18.777310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f9fad-761d-4944-a19b-df39bab5c882 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:18.778110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f9fad-761d-4944-a19b-df39bab5c882 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:18.778251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f9fad-761d-4944-a19b-df39bab5c882 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:18.779721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 24/47] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:18.785725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:18.786183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:18.786433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:18.810340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:18.811947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:18.888383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 24/48] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:19.172760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f9fad-761d-4944-a19b-df39bab5c882 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:19.293679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:19.300792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f9fad-761d-4944-a19b-df39bab5c882 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:19.339802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:19.340581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:19.340959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:19.341318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:19.351874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:19.356834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:19.357786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:19.358534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 25/49] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:19.364430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:19.424519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:19.473874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ea510-7fe5-4226-b238-185c0c381f33 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:19.509125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:19.509865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:19.510051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:19.510309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:19.510992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:19.511290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:19.512742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:19.522755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 25/50] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:19.528486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:19.637166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824bd3e-a116-4af0-874e-f2a81886cdf5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:19.673970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1781e-6351-4637-a364-ac8af9f4d4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:19.674685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1781e-6351-4637-a364-ac8af9f4d4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:19.674899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1781e-6351-4637-a364-ac8af9f4d4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:19.675172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1781e-6351-4637-a364-ac8af9f4d4e9 None admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:19.676389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1781e-6351-4637-a364-ac8af9f4d4e9 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:19.678325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1781e-6351-4637-a364-ac8af9f4d4e9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:19.680198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 26/51] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:19 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:19.685856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:19.795726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c372e6-bbec-4559-bd88-6b41389f5c4b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:19.913408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e8ef9-f849-4453-ba2f-6f566c7672e1 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:19.913926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e8ef9-f849-4453-ba2f-6f566c7672e1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:19.914013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e8ef9-f849-4453-ba2f-6f566c7672e1 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:19.915442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 27/52] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:19.920607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:19.920903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:19.921207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:19.923790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:19.938952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:19.938952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:19.939238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:19.939423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:19.940379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:19.940587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:19.941748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:19.942075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:19.943683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 26/53] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:19.949749 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:19.960445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:20.017078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 28/54] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:20.071805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460ef0e-f7dc-4424-8da3-369fa7d97478 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:20.128405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:20.129429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:20.129921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:20.130382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:20.131679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.domain.id=default, user.id=6fd5eb0c7a1f4bdbb078438d56380575) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:20.132298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:20.135063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:20.159915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 27/55] 10.222.0.65 () {56 vars in 1322 bytes} [Mon Aug 29 21:03:19 2022] GET /identity/v3/role_assignments?user.id=6fd5eb0c7a1f4bdbb078438d56380575&scope.domain.id=default&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 530 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:20.288719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:20.392859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c5bfb-eb57-46b7-b728-0d35431fa4c7 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:20.426893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b874955-94fd-483b-8f61-615d06ca36e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:20.427527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b874955-94fd-483b-8f61-615d06ca36e0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:20.427810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b874955-94fd-483b-8f61-615d06ca36e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:20.428221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b874955-94fd-483b-8f61-615d06ca36e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:20.429352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b874955-94fd-483b-8f61-615d06ca36e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:20.430815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b874955-94fd-483b-8f61-615d06ca36e0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:20.432415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 29/56] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:20.438043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:20.492360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:20.539529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00078f-410c-495c-b29e-3c57d1f80b77 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:20.572069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113064f-b127-49d3-a4ba-aa17171233de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:20.572679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113064f-b127-49d3-a4ba-aa17171233de None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:20.572963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113064f-b127-49d3-a4ba-aa17171233de None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:20.573236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113064f-b127-49d3-a4ba-aa17171233de None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:20.573895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113064f-b127-49d3-a4ba-aa17171233de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:20.575355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113064f-b127-49d3-a4ba-aa17171233de None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:20.679574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 28/57] 10.222.0.65 () {60 vars in 1140 bytes} [Mon Aug 29 21:03:20 2022] POST /identity/v3/projects => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:22.602738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70d3ac-e67b-4fd6-b851-711492991502 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:22.603254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70d3ac-e67b-4fd6-b851-711492991502 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:22.603342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70d3ac-e67b-4fd6-b851-711492991502 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:22.605029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 30/58] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:22.613038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:22.613680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:22.613680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:22.631705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:22.632962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:22.703984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 29/59] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:22.978686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70d3ac-e67b-4fd6-b851-711492991502 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:23.001526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:23.085636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70d3ac-e67b-4fd6-b851-711492991502 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:23.120636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b1525-ecd8-440b-bf6f-c75cc7376b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:23.121345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b1525-ecd8-440b-bf6f-c75cc7376b13 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:23.121671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b1525-ecd8-440b-bf6f-c75cc7376b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:23.121988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b1525-ecd8-440b-bf6f-c75cc7376b13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:23.123178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b1525-ecd8-440b-bf6f-c75cc7376b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:23.125314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b1525-ecd8-440b-bf6f-c75cc7376b13 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:23.127264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 31/60] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:23.134063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:23.167306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:23.247149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7f92f-c628-43fa-b7b2-863102e8f8b0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:23.285135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845de323-294a-42bd-a127-87d2a7ced1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:23.285774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845de323-294a-42bd-a127-87d2a7ced1bb None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:23.286054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845de323-294a-42bd-a127-87d2a7ced1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:23.286335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845de323-294a-42bd-a127-87d2a7ced1bb None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:23.287013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845de323-294a-42bd-a127-87d2a7ced1bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:23.288542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845de323-294a-42bd-a127-87d2a7ced1bb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:23.422307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 30/61] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:03:23 2022] POST /identity/v3/users => generated 275 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:03:26.820707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7706df-8428-4971-9c7f-d0ec383f2c44 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:26.821878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7706df-8428-4971-9c7f-d0ec383f2c44 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:26.822388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7706df-8428-4971-9c7f-d0ec383f2c44 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:26.824124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 32/62] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:26.830795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d09a-891e-4814-821b-f3d213d05819 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:26.831728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d09a-891e-4814-821b-f3d213d05819 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:26.832347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d09a-891e-4814-821b-f3d213d05819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:26.860973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-83d1d09a-891e-4814-821b-f3d213d05819 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:26.863027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-83d1d09a-891e-4814-821b-f3d213d05819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:26.958179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 31/63] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:27.199119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2393d-fbc5-4988-91f0-155b5e9cf904 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.199782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2393d-fbc5-4988-91f0-155b5e9cf904 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.200277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2393d-fbc5-4988-91f0-155b5e9cf904 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.201843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 33/64] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:27.207809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff3b9-a115-4849-9d38-7d7318821868 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.208372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff3b9-a115-4849-9d38-7d7318821868 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.208845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff3b9-a115-4849-9d38-7d7318821868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.231243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2393d-fbc5-4988-91f0-155b5e9cf904 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:27.246646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b96ff3b9-a115-4849-9d38-7d7318821868 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:27.249463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b96ff3b9-a115-4849-9d38-7d7318821868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:27.276518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:27.395577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:27.396672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 32/65] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:27.399084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f8330-bcad-4728-804a-9094f5360367 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.399634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f8330-bcad-4728-804a-9094f5360367 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.400158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f8330-bcad-4728-804a-9094f5360367 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.401743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 33/66] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:27.403074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580a0c-0baf-40bb-a480-003f8f82f071 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.411283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580a0c-0baf-40bb-a480-003f8f82f071 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.411283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580a0c-0baf-40bb-a480-003f8f82f071 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.411283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 34/67] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:27.412986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.414008 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.414689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.415318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2393d-fbc5-4988-91f0-155b5e9cf904 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:27.433774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:27.435153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:27.459565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc8c31-bf70-4baf-add6-0d6fbe2e8dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:27.460789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc8c31-bf70-4baf-add6-0d6fbe2e8dbd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.461128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc8c31-bf70-4baf-add6-0d6fbe2e8dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.461488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc8c31-bf70-4baf-add6-0d6fbe2e8dbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.474831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc8c31-bf70-4baf-add6-0d6fbe2e8dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-34cc8c31-bf70-4baf-add6-0d6fbe2e8dbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:27.481155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:27.482126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:27.482956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:27.486164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 34/68] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:27.488136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b220462-388b-47cc-9349-2dd7799f9262 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.488619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b220462-388b-47cc-9349-2dd7799f9262 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.489059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b220462-388b-47cc-9349-2dd7799f9262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.512686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-1b220462-388b-47cc-9349-2dd7799f9262 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:27.514267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1b220462-388b-47cc-9349-2dd7799f9262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:27.534351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 35/69] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:27.541870 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:27.640678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 35/70] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:27.659110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4710022-2e64-4625-9d31-e2e4cc446b78 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.659287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4710022-2e64-4625-9d31-e2e4cc446b78 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.659572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4710022-2e64-4625-9d31-e2e4cc446b78 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.661299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 36/71] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:27.667354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.667669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.667967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.696571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:27.699645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:27.730228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:27.755427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e818e5-5209-4f15-af1d-35ff706a830f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:27.805761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:27.807038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.807717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:27.808142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:27.809685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:27.810103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:27.812718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:27.816044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 37/72] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:27.816906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:27.829811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 36/73] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:27.833039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:27.836881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:27.961004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade9bed-7ac4-43a6-b480-e52ba0da8b00 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:27.978580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd49df-9a57-4015-bda0-af17cb30f06e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:27.998592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:27.999276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:27.999563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.000282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.011555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:28.015593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.016276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.016562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.016843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:28.017144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:28.017995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:28.018753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 38/74] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:28.019380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.035729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.038738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.040274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.046669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 37/75] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:27 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:28.046669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.103307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.140762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a25d7e-f876-4d67-86da-eadf0577ea5c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.162063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eeeba-e0c6-487a-9a04-bce1dab509f8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.178797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.179324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.179604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.179893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.190530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:28.195517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:28.196430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:28.197268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 39/76] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:28.197915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.198007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.198078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.198078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.198078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.211558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:28.217898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:28.219379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:28.220177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 38/77] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:28.221886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.291797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.310689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24fa70-e112-4979-9728-856822d99bc6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.333553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d6f0b-4aee-4e85-a9d3-6e13dd175107 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.338014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.346042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.346666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.346956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.347236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.347988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:28.348498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.349780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.358385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 40/78] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:28.363632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.371208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.371831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.372148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.372427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.448609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.450124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.474718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac788-1342-416a-97d9-65adb41ab900 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.509720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.510370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.510657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.510947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.511171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.512109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.513591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.515978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 41/79] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:28.516650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.557648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 39/80] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:28.559263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.587994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.637271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bca3d-e801-4146-a017-093afeb059b7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.673717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a63af4-cf0d-48c4-847b-5817c273ff25 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.677079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.677728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.678007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.678278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.678942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:28.679247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.680698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.689805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 42/81] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:28.690649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.714443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.715063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.715346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.715627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.716592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:28.716712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.718048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.726476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 40/82] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:28.727317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.755135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.796462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75554b45-b04e-4609-90c4-c6fd1b64d8b0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.821891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:28.842252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.842900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b225a11-333c-4850-82b2-dbcab3ee3e63 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.843519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.844327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.845157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.846698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.849301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.852084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 43/83] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:28.853351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:28.898429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:28.899049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:28.899326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:28.899603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:28.900298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:28.901776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:28.964033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bd545-815a-476d-84b5-a9e510a1e975 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:28.999038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:29.000186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:29.000469 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:29.000745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:29.001959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:29.003457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:29.005166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 44/84] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:28 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:29.005954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:29.014007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 41/85] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:03:28 2022] POST /identity/v3/users => generated 289 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:29.014895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:29.106275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:29.152219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:29.179156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510f83e-7cb4-49c5-9373-2618e825b209 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:29.180846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cf71b-8978-4ea8-b96b-2e37b71692ac None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:29.229397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:29.230286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:29.230751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:29.231033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:29.234688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:29.235017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:29.237353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:29.237724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 45/86] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:29 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:29.238897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:29.239395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:29.239775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:29.242707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:29.243924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:29.244033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:29.246983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 42/87] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:29 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:29.251265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:29.405518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8b09c-1dcb-4834-b7a4-a457af486b78 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:29.408424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efadcae-ea14-43ab-b480-76882b2fa1c5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:29.442654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:29.451046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:29.452129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:29.452612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:29.453200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:29.453583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:29.454410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:29.454737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:29.455005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:29.455822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=7e902eff1ce140c9a81aac150e114ae7) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:29.456171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:29.457688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:29.465844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 43/88] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:29 2022] GET /identity/v3/role_assignments?user.id=7e902eff1ce140c9a81aac150e114ae7&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=87804c58f0314370a20ec138295f6863 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:29.466839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:29.487922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:29.621726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:29.623269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:29.649558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf1cfa-c858-46f6-9421-973d209f4354 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:29.713232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:29.714506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:29.715280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:29.715972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:29.837704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 46/89] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:29.839339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:29.873022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:29.893526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:29.897102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:29.928430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:30.002368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5bb56-0456-4665-b428-cbdd1ae5b6bc None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.042169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.043275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.043824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.044273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.145549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.146289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 44/90] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:30.147071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:30.147173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.179908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:30.261952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 47/91] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:30.262652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:30.270462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548fa2-dfec-4046-a9dc-c59828a66978 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.304329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:30.312607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.313244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.313537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.313823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.315037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.316593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.318301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 45/92] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:30 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:30.319212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:30.378130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d647c-5c64-4872-8a86-1095c6daab3f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.403871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:30.424636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.425360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.425654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.425938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.427259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.428760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.430468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 48/93] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:30 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:30.431243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:30.504691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f486e-333f-4553-a8c8-4c9f20bd2a74 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.541575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.542258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.542543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.542824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] PATH_INFO: `/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.544149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] RBAC: Authorizing `identity:get_project(project_id=f9592ce6b5e44e00be8e402c2ab7dc65)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.545663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.547583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 46/94] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:30 2022] GET /identity/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:30.548229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:30.557377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e487f-5544-4b1b-8d4f-5bc3d27e3205 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.597863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.598432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:30.599262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.599551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.599835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.600613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=6fd5eb0c7a1f4bdbb078438d56380575) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:30.600932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.602439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.608892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 49/95] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:30 2022] GET /identity/v3/role_assignments?user.id=6fd5eb0c7a1f4bdbb078438d56380575&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:30.609732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:30.659741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b1546-0a14-4f02-9b96-23dd4c3f54cf None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.701675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.702865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.703158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.703436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.704211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] RBAC: Adding query filter params (role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=7e902eff1ce140c9a81aac150e114ae7) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:30.704524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.705998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5916e-6d8a-4373-a187-06641fc4e632 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.711918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 47/96] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:30 2022] GET /identity/v3/role_assignments?user.id=7e902eff1ce140c9a81aac150e114ae7&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:30.747354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:30.854707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff51d55-d6a7-4b7f-b3bc-d8aa32d9a62f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:30.920177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:30.921440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:30.921647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:30.921902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:30.923195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, scope.project.id=f9592ce6b5e44e00be8e402c2ab7dc65, user.id=7e902eff1ce140c9a81aac150e114ae7) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:30.923566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:30.925116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af763163-90f9-43da-a76a-05a0b2bcea91 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:30.933804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 50/97] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:30 2022] GET /identity/v3/role_assignments?user.id=7e902eff1ce140c9a81aac150e114ae7&scope.project.id=f9592ce6b5e44e00be8e402c2ab7dc65&role.id=87804c58f0314370a20ec138295f6863 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:31.780965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6a07-5898-4ea8-9b44-4b5bb7a5c28a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:31.781538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6a07-5898-4ea8-9b44-4b5bb7a5c28a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:31.781538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe6a07-5898-4ea8-9b44-4b5bb7a5c28a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:31.782905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 48/98] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:31.788820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5adc6-1301-4ba9-98d4-1b683d3f31f2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:31.789096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5adc6-1301-4ba9-98d4-1b683d3f31f2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:31.789374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5adc6-1301-4ba9-98d4-1b683d3f31f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:31.817319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-55c5adc6-1301-4ba9-98d4-1b683d3f31f2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:31.819070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-55c5adc6-1301-4ba9-98d4-1b683d3f31f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:31.924687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 51/99] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:32.297461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd605e-06e8-40dd-8ec4-9be471db7c39 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:32.297785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd605e-06e8-40dd-8ec4-9be471db7c39 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:32.298090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd605e-06e8-40dd-8ec4-9be471db7c39 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:32.299817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 49/100] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:32.306344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3bd9d-532a-4621-9c41-8cd35e3966e5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:32.306696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3bd9d-532a-4621-9c41-8cd35e3966e5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:32.306951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3bd9d-532a-4621-9c41-8cd35e3966e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:32.337197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-93a3bd9d-532a-4621-9c41-8cd35e3966e5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:32.338451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-93a3bd9d-532a-4621-9c41-8cd35e3966e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:32.410816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:32.438421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 52/101] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:32.634588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e845567-b7a0-4a33-96d3-101b73d26b6b None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:32.635018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e845567-b7a0-4a33-96d3-101b73d26b6b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:32.635186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e845567-b7a0-4a33-96d3-101b73d26b6b None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:32.636567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 50/102] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:32.642427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:32.642721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:32.643003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:32.647232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e845567-b7a0-4a33-96d3-101b73d26b6b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:32.694334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:32.696047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:32.874202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e845567-b7a0-4a33-96d3-101b73d26b6b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:32.875404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 53/103] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:32.897085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:32.916380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:32.956620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:32.958658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:32.959595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:32.960521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:32.962784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:32.973578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:32.976421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 51/104] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:32.985235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.132330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f649586-286d-4f90-866c-cb3a7f050b0b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.148338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:33.172638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.173292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.173721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.174007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.186695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:33.192275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:33.193197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:33.194020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 54/105] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:33.204855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.245442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63227d3-3436-4128-a0bd-93e3fab1b7b5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.316253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1ca1a-44d1-437c-b76c-ca23a1caaf64 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.344595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.350880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.350880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.350880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.350880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:33.350880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:33.361837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.362682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.362980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.363260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.363960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:33.364274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:33.365698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:33.384056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 55/106] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:33.385016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.424309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:33.497971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa9d2a-1f9a-4e43-8932-8d72111e216c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.541509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 52/107] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:03:32 2022] POST /identity/v3/projects => generated 302 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:03:33.542452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.547441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.548201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.548611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.548898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.563568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:33.569819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:33.571524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:33.573124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 56/108] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:33.578307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.633630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:33.706188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543aba66-d25f-47ab-b38d-12b17034a3b4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.712331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:33.787688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.788528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.789107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.789662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.790972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:33.791623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:33.794455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf74c1-b10f-40e3-97da-a2d36e46b1c0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:33.806969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9c547-2866-44e1-b362-0784784406b5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.809841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 57/109] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:33.811730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d896662-1985-43c2-94fa-5cb5cd0984b1 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.812115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d896662-1985-43c2-94fa-5cb5cd0984b1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.812460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d896662-1985-43c2-94fa-5cb5cd0984b1 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.813849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 58/110] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:33.819205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d896662-1985-43c2-94fa-5cb5cd0984b1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.872597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c554c-cfb0-48b0-a340-b642798f9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.873268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c554c-cfb0-48b0-a340-b642798f9cfd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.874081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c554c-cfb0-48b0-a340-b642798f9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.874366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c554c-cfb0-48b0-a340-b642798f9cfd None admin] PATH_INFO: `/v3/users/5f3566fb0a7c451cb5375b1861e939b7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.892074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c554c-cfb0-48b0-a340-b642798f9cfd None admin] RBAC: Authorizing `identity:get_user(user_id=5f3566fb0a7c451cb5375b1861e939b7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:33.894211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c554c-cfb0-48b0-a340-b642798f9cfd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:33.896829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 53/111] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:33 2022] GET /identity/v3/users/5f3566fb0a7c451cb5375b1861e939b7 => generated 289 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:33.898159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.898452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.898751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.919148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:33.920962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:33.949521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d896662-1985-43c2-94fa-5cb5cd0984b1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:33.991551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:33.992674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:33.993265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:33.993823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:33.995341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:33.995950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 54/112] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:33.996908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:33.997823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:34.004902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 59/113] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:33 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:34.010554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:34.017718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:34.054139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:34.121453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da83b8-2c4e-4411-b10e-73d5cc8c7061 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:34.186201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:34.187027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.187323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.187607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.227367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2ac4f-60b7-4cf1-9532-0c92394b6714 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:34.282004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:34.284075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:34.307150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:34.307935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.308331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.308543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:34.308806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.310245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:34.312649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:34.315108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 60/114] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:34 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:34.321113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:34.393871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 55/115] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:34.399572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:34.469412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:34.480279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f49a3b-d7d3-4bee-83a7-c990c10d62fe None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:34.510062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10fb1e-34c6-4153-967e-871273362ade None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:34.524554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ea5b4-65b9-40a1-b06e-3ef7ae2973d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:34.525647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ea5b4-65b9-40a1-b06e-3ef7ae2973d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.526194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ea5b4-65b9-40a1-b06e-3ef7ae2973d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.526674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ea5b4-65b9-40a1-b06e-3ef7ae2973d1 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118/users/7e902eff1ce140c9a81aac150e114ae7/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.529091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ea5b4-65b9-40a1-b06e-3ef7ae2973d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=263e684247604a1a8e174c46e3574118, user_id=7e902eff1ce140c9a81aac150e114ae7, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:34.529769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:34.531772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ea5b4-65b9-40a1-b06e-3ef7ae2973d1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:34.548655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dabc9-28c6-41ab-a88b-a5dd458805c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:34.549314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dabc9-28c6-41ab-a88b-a5dd458805c8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.549602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dabc9-28c6-41ab-a88b-a5dd458805c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.549882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dabc9-28c6-41ab-a88b-a5dd458805c8 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.551071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dabc9-28c6-41ab-a88b-a5dd458805c8 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:34.552614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dabc9-28c6-41ab-a88b-a5dd458805c8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:34.557580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 56/116] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:34 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:34.560432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff2047-9c45-41ed-893d-2daaa593fc14 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.560432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff2047-9c45-41ed-893d-2daaa593fc14 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.563176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff2047-9c45-41ed-893d-2daaa593fc14 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.563176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 57/117] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:34.564759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e4040-1e69-41de-bd66-6f8f18c1ae84 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.565195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e4040-1e69-41de-bd66-6f8f18c1ae84 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.565623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e4040-1e69-41de-bd66-6f8f18c1ae84 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.569960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 58/118] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:34.571562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e4040-1e69-41de-bd66-6f8f18c1ae84 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:34.629595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 61/119] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:34 2022] PUT /identity/v3/projects/263e684247604a1a8e174c46e3574118/users/7e902eff1ce140c9a81aac150e114ae7/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:03:34.631616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc6ceb-fda9-42de-9ec9-af70c3863781 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.632078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc6ceb-fda9-42de-9ec9-af70c3863781 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.632549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc6ceb-fda9-42de-9ec9-af70c3863781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.662102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d1dc6ceb-fda9-42de-9ec9-af70c3863781 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:34.664758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d1dc6ceb-fda9-42de-9ec9-af70c3863781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:34.752683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e4040-1e69-41de-bd66-6f8f18c1ae84 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:34.777190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 62/120] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:34.783404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.783872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.784391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.795992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:34.797066 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:34.797584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:34.798060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:34.799018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] RBAC: Adding query filter params (role.id=a84e6592db084ce2b9667f2cd70fdcb2, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=5f3566fb0a7c451cb5375b1861e939b7) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:34.799597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:34.801535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:34.808388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 59/121] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:34 2022] GET /identity/v3/role_assignments?user.id=5f3566fb0a7c451cb5375b1861e939b7&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=a84e6592db084ce2b9667f2cd70fdcb2 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:34.817359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:34.826012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:34.832394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:34.920097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:34.965227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 63/122] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:35.005622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5416f7-5e7b-4fa2-be2b-6703570aed59 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:35.069420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:35.070166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:35.070459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:35.070804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:35.091080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:35.102807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:35.104390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:35.105829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 60/123] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:03:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:35.114282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:35.136430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:35.265732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:35.291193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb109c-45d1-41a5-9b9a-490791c5a7a3 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:35.409030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:35.409619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:35.409910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:35.410647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:35.410887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:35.411172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:35.412640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:35.445093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 64/124] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:03:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:35.461626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:35.503915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d52997-a78a-4c1a-80b2-fcd8742c2f2e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:35.524760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:35.578441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:35.579107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:35.579393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:35.579671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:35.591790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:35.597143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:35.598138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 61/125] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:35.599665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:35.659297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:35.685531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17eb91-eed8-440d-be06-412b08c341a2 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:35.744888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:35.745995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:35.746711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:35.747410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:35.748464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886abde-1110-4b0b-9e4f-c25b63e98853 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:35.749722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:35.751772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:35.760034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 65/126] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:35 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:35.761577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:35.813430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:35.814337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:35.814679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:35.815050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:35.831857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:35.840091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:35.841686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:35.842975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 62/127] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:35.844210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:35.921068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:35.948846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d929c-3c7a-4167-89c2-2a0b2582cdb9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:35.986679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:35.987671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:35.988304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:35.988755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:35.989806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:35.990359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:35.991968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.001334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 66/128] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:36.003004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.016425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:36.115845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920e1ec-a11f-4eea-a62a-0cd220460120 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.164252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.165496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.166040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.166709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.167860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:36.168454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.169985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb615a-f1a5-4224-8e46-b6136826c73f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.170991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.186626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 67/129] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:36.188369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.276465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:36.307735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f0ab7-7765-4ef8-8994-d0892152c217 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.317968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.318644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.318933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.319211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.324621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.326119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.329189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 63/130] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:35 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:36.330055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.357435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.359738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.360161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.360510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.362246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.363870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.366069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 68/131] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:36 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:36.366921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.480033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c1463-3da3-41cf-8084-1b5a3e22c766 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.525625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:36.534789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.539233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.539233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.539233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118/users/7e902eff1ce140c9a81aac150e114ae7/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.540702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] RBAC: Authorizing `identity:create_grant(project_id=263e684247604a1a8e174c46e3574118, user_id=7e902eff1ce140c9a81aac150e114ae7, role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.542184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.599077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6866a00-4406-470a-8a7d-8061ddfa5a1e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.613178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:36.634551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 69/132] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:36 2022] PUT /identity/v3/projects/263e684247604a1a8e174c46e3574118/users/7e902eff1ce140c9a81aac150e114ae7/roles/c2a3834a436b40deabcb87f412ca5322 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:03:36.640661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.643570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.644312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.644635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.644912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.647665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.650559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.654885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 64/133] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:36 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:36.668395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.884268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971b732-4b60-4153-bf95-b8d4e9778b33 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.890700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cf444-45e6-4ea9-b80f-5cea488db028 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:36.952258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.953255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.953255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.953509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] PATH_INFO: `/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.954772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] RBAC: Authorizing `identity:get_project(project_id=f9592ce6b5e44e00be8e402c2ab7dc65)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.956455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.958253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 65/134] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:36 2022] GET /identity/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65 => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:36.964715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:36.968319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:36.969100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:36.969393 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:36.969677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:36.974992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:36.976707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:36.978895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 70/135] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:36 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:36.984741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:37.007866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:37.092680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fb4a8-d7ef-4c4c-af3f-1b7c89d442d0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:37.135745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d56e12-bb1e-493c-be6e-f4b67ffe1ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:37.136484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d56e12-bb1e-493c-be6e-f4b67ffe1ec1 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.136772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d56e12-bb1e-493c-be6e-f4b67ffe1ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.137052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d56e12-bb1e-493c-be6e-f4b67ffe1ec1 None admin] PATH_INFO: `/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65/users/7e902eff1ce140c9a81aac150e114ae7/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.139554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d56e12-bb1e-493c-be6e-f4b67ffe1ec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9592ce6b5e44e00be8e402c2ab7dc65, user_id=7e902eff1ce140c9a81aac150e114ae7, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:37.141172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d56e12-bb1e-493c-be6e-f4b67ffe1ec1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:37.153973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:37.214349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 66/136] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:36 2022] PUT /identity/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65/users/7e902eff1ce140c9a81aac150e114ae7/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:03:37.267844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c6d59-f575-45ae-933c-ef1abb68e979 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:37.357202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129ee38-9f1b-4ab0-81c7-dc1e7e767282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:37.358945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ee38-9f1b-4ab0-81c7-dc1e7e767282 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.359448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ee38-9f1b-4ab0-81c7-dc1e7e767282 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.359997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129ee38-9f1b-4ab0-81c7-dc1e7e767282 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118/users/6fd5eb0c7a1f4bdbb078438d56380575/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.363909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129ee38-9f1b-4ab0-81c7-dc1e7e767282 None admin] RBAC: Authorizing `identity:create_grant(project_id=263e684247604a1a8e174c46e3574118, user_id=6fd5eb0c7a1f4bdbb078438d56380575, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:37.367667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129ee38-9f1b-4ab0-81c7-dc1e7e767282 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:37.458919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 71/137] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:36 2022] PUT /identity/v3/projects/263e684247604a1a8e174c46e3574118/users/6fd5eb0c7a1f4bdbb078438d56380575/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:03:37.459505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb544d-0403-49cc-bdeb-2373e6093b6d None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.460041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb544d-0403-49cc-bdeb-2373e6093b6d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.460484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb544d-0403-49cc-bdeb-2373e6093b6d None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.463632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 67/138] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:37.470087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae80a45-25d6-405e-a182-f3c9f6589f13 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.470589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae80a45-25d6-405e-a182-f3c9f6589f13 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.472404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae80a45-25d6-405e-a182-f3c9f6589f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.517849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0ae80a45-25d6-405e-a182-f3c9f6589f13 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:37.519913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0ae80a45-25d6-405e-a182-f3c9f6589f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:37.618571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ff09c-d609-46e1-aff6-e1597e6468ec None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.619021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ff09c-d609-46e1-aff6-e1597e6468ec None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.619263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ff09c-d609-46e1-aff6-e1597e6468ec None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.620508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 68/139] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:37.628289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21667234-d3f5-4099-bf6d-ea02ff45599c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.628605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21667234-d3f5-4099-bf6d-ea02ff45599c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.628896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21667234-d3f5-4099-bf6d-ea02ff45599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.666686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-21667234-d3f5-4099-bf6d-ea02ff45599c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:37.669007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 72/140] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:37.674812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-21667234-d3f5-4099-bf6d-ea02ff45599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:37.824530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 69/141] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:37.883170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3a4b8-3fc5-42fc-a6d3-ca66b57aa5a2 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.883631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3a4b8-3fc5-42fc-a6d3-ca66b57aa5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.884161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3a4b8-3fc5-42fc-a6d3-ca66b57aa5a2 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.886675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 73/142] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:37.895486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:37.896010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:37.896511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:37.927136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:37.929278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:37.961988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:38.069295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 70/143] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:38.300659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3a4b8-3fc5-42fc-a6d3-ca66b57aa5a2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:38.352427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:38.381991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:38.535748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3a4b8-3fc5-42fc-a6d3-ca66b57aa5a2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:38.603289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:38.604436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:38.604948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:38.605536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:38.629714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:38.640784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:38.641646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:38.642406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:38.643486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 74/144] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:38.644749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:38.690171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76153b3-b1cc-4ac3-a2e4-a87bef5b97ce None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:38.795822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:38.796645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:38.796938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:38.797216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:38.825512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:38.830571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:38.831500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:38.832301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 71/145] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:38.832949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:38.836040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c6a2-1440-4281-b2a4-b93ecdfd95aa None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:38.849461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:38.870019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:38.878504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:38.879273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:38.879673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:38.880044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:38.881288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:38.883480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:38.885564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 75/146] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:38.886912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:39.062261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453b95-2146-4e7e-895d-dabffa57b9cf None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:39.067292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68ad60-5c03-4184-a073-4278363e401a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:39.114279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:39.115398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:39.115862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:39.116344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:39.117804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:39.118319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:39.120231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:39.145198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 76/147] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:39.148313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:39.162147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:39.163175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:39.163678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:39.164003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:39.164697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:39.165005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:39.166415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:39.183209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 72/148] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:39.184111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:39.235216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:39.328974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf6645-6d0e-4adb-92ad-479fe063f170 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:39.346620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:39.375712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:39.377096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:39.377555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:39.378002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:39.378886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:39.381352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:39.400832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d879de-8d21-428f-8dba-6ad061ffe25a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:39.478353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 77/149] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:03:39 2022] POST /identity/v3/users => generated 283 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:39.482973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:39.484572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:39.488185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:39.489141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:39.490423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:39.496324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:39.501979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:39.514415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 73/150] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:39 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:39.516571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:39.712470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:39.716475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ba5fb-9c96-4a77-8f9b-0bb2b11fefe9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:39.766675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:39.767668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:39.769127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:39.769609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] PATH_INFO: `/v3/users/5f3566fb0a7c451cb5375b1861e939b7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:39.771105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] RBAC: Authorizing `identity:get_user(user_id=5f3566fb0a7c451cb5375b1861e939b7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:39.772926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:39.775417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 78/151] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:39 2022] GET /identity/v3/users/5f3566fb0a7c451cb5375b1861e939b7 => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:39.784340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:39.835153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464dbd8-1c26-462f-85f4-613d59160e13 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:39.913715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a660e9db-2506-460f-9b3a-bc0509a2404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:39.913715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660e9db-2506-460f-9b3a-bc0509a2404d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:39.913715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660e9db-2506-460f-9b3a-bc0509a2404d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:39.914832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660e9db-2506-460f-9b3a-bc0509a2404d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:39.935448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:39.953204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620b171-99d9-44a1-b8fc-af01d72c9c32 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:40.003650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c316010-0aee-4449-a925-145c48ffc732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:40.004916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c316010-0aee-4449-a925-145c48ffc732 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.005379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c316010-0aee-4449-a925-145c48ffc732 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.005844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c316010-0aee-4449-a925-145c48ffc732 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.009616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c316010-0aee-4449-a925-145c48ffc732 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:40.012306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c316010-0aee-4449-a925-145c48ffc732 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:40.014470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 79/152] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:39 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:40.015622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137e73-5339-4770-a1c6-9c57ea0f61f1 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.015961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137e73-5339-4770-a1c6-9c57ea0f61f1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.016266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d137e73-5339-4770-a1c6-9c57ea0f61f1 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.019617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 80/153] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:40.020266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d137e73-5339-4770-a1c6-9c57ea0f61f1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:40.034576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660e9db-2506-460f-9b3a-bc0509a2404d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:40.037166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a660e9db-2506-460f-9b3a-bc0509a2404d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:40.082172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:40.186559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 74/154] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:40.187753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d137e73-5339-4770-a1c6-9c57ea0f61f1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:40.188528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d488-4f16-4550-8ec3-3a6bdb9d782b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.188882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d488-4f16-4550-8ec3-3a6bdb9d782b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.189438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d488-4f16-4550-8ec3-3a6bdb9d782b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.223642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e490d488-4f16-4550-8ec3-3a6bdb9d782b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:40.227836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e490d488-4f16-4550-8ec3-3a6bdb9d782b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:40.241415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e1eb7-a279-43fa-a52f-3992b9d518cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:40.242111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1eb7-a279-43fa-a52f-3992b9d518cb None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.242504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1eb7-a279-43fa-a52f-3992b9d518cb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.242783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e1eb7-a279-43fa-a52f-3992b9d518cb None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118/users/5f3566fb0a7c451cb5375b1861e939b7/roles/a84e6592db084ce2b9667f2cd70fdcb2` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.246163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e1eb7-a279-43fa-a52f-3992b9d518cb None admin] RBAC: Authorizing `identity:create_grant(project_id=263e684247604a1a8e174c46e3574118, user_id=5f3566fb0a7c451cb5375b1861e939b7, role_id=a84e6592db084ce2b9667f2cd70fdcb2)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:40.248545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e1eb7-a279-43fa-a52f-3992b9d518cb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:40.284414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:40.315573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 75/155] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:40.317315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86423-fc38-494c-a18f-213369900445 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.317708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86423-fc38-494c-a18f-213369900445 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.318094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf86423-fc38-494c-a18f-213369900445 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.320291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 76/156] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:40.323181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf86423-fc38-494c-a18f-213369900445 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:40.333551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 81/157] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:40 2022] PUT /identity/v3/projects/263e684247604a1a8e174c46e3574118/users/5f3566fb0a7c451cb5375b1861e939b7/roles/a84e6592db084ce2b9667f2cd70fdcb2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:03:40.335108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d843c3-389a-4be1-ac3f-35ee03c64cc7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.335560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d843c3-389a-4be1-ac3f-35ee03c64cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.336023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d843c3-389a-4be1-ac3f-35ee03c64cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.372714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e3d843c3-389a-4be1-ac3f-35ee03c64cc7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:40.374396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e3d843c3-389a-4be1-ac3f-35ee03c64cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:40.518778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 82/158] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:40.552647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf86423-fc38-494c-a18f-213369900445 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:40.574650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e1f5e-b08c-446f-b14d-2bfd9640bb68 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.575552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e1f5e-b08c-446f-b14d-2bfd9640bb68 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.575987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e1f5e-b08c-446f-b14d-2bfd9640bb68 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.583218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 83/159] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:40.586573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.587251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.587740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.624080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:40.624762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.625044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.625326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.626627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:40.628226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:40.628351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:40.630141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 77/160] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:40 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:40.630274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:40.636239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:40.685202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:40.723845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:40.733050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 84/161] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:40.755007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:40.858812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad6cf3-2a00-4cb1-9a8e-8f10aface382 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:40.921223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:40.921847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:40.922130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:40.922412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:40.923807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=7e902eff1ce140c9a81aac150e114ae7) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:40.924152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:40.926537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:40.938172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 78/162] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:40 2022] GET /identity/v3/role_assignments?user.id=7e902eff1ce140c9a81aac150e114ae7&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=87804c58f0314370a20ec138295f6863 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:40.979091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e72bf7-c577-47bf-8d01-10949cdabee2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:41.044243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:41.046516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:41.047239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:41.047853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:41.060009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:41.081126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:41.088625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:41.089898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:41.090978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:41.098167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 85/163] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:41.105719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:41.146847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:41.260572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c149b43-e8cd-44a1-901e-b7ea064be2ee None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:41.276077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:41.352684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:41.353357 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:41.353665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:41.353913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:41.381154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:41.386092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:41.386958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:41.387739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:41.390670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 79/164] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:41.396080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:41.424340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d6a32-d8f9-4d5b-a169-c92152011480 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:41.498575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:41.499482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:41.499762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:41.500079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:41.500927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:41.501237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:41.502666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:41.517984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 86/165] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:03:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:41.518867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:41.657913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc15432-edb5-4c56-ba18-52fa11168e0f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:41.665485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:41.716391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb0c5-4b5e-469d-9210-4dd270016642 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:41.745366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:41.746662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:41.747288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:41.748419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:41.750148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:41.751091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:41.754112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:41.775695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 80/166] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:41.777201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:41.807773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:41.808652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:41.808941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:41.809214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:41.837319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:41.840455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:41.843279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:41.844195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:41.845074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:41.857701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 87/167] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:41.858654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:42.026491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6412e2f-b3d6-4adf-a648-e7d0d7a2bc3e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:42.107556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39f0d2-b319-4ff0-9388-3b2463293e2e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:42.119251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:42.120239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:42.120658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:42.121046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:42.123073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:42.129170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:42.136342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 81/168] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:41 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:42.137310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:42.210116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:42.210801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:42.211086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:42.211367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] PATH_INFO: `/v3/users/7e902eff1ce140c9a81aac150e114ae7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:42.212685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] RBAC: Authorizing `identity:get_user(user_id=7e902eff1ce140c9a81aac150e114ae7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:42.214232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:42.220040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 88/169] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:41 2022] GET /identity/v3/users/7e902eff1ce140c9a81aac150e114ae7 => generated 275 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:42.221172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:42.226218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:42.343206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:42.355619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c64636-3eaa-4c52-b316-f238b69a0d83 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:42.412495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c8099-7174-4eab-94fe-315460d81537 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:42.444016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:42.444996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:42.445321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:42.445749 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:42.446630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:42.447110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:42.449148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:42.470028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 82/170] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:42.472617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:42.507539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:42.508215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:42.508499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:42.508781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:42.666724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:42.666724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:42.667988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:42.699442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25408650-ebec-4d96-abe2-aa9c5b2ab891 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:42.778067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:42.778805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:42.779004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:42.779278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:42.837398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:42.885203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 89/171] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:42.886163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:42.941054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:42.942517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:43.082284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5c74a-3e36-4183-aba2-ace0cd5c2377 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:43.084473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:43.168818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:43.169101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 83/172] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:43.169805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:43.170637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:43.171335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:43.171983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] PATH_INFO: `/v3/users/6fd5eb0c7a1f4bdbb078438d56380575` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:43.177154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] RBAC: Authorizing `identity:get_user(user_id=6fd5eb0c7a1f4bdbb078438d56380575)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:43.180820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:43.187914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 90/173] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:42 2022] GET /identity/v3/users/6fd5eb0c7a1f4bdbb078438d56380575 => generated 247 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:43.193326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:43.308424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:43.409987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d27c1-5d74-4ad5-8484-a0f784219dbb None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:43.486544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9ba1a-1697-4382-8d7a-445992673867 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:43.534759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:43.535875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:43.536398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:43.536896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:43.541060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:43.546609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:43.557700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 84/174] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:43 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:43.561582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:43.605516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93340d-dc92-40c3-b66d-5aa16ffe2947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:43.606593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93340d-dc92-40c3-b66d-5aa16ffe2947 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:43.607113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93340d-dc92-40c3-b66d-5aa16ffe2947 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:43.607615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93340d-dc92-40c3-b66d-5aa16ffe2947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:43.741142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:43.792467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93340d-dc92-40c3-b66d-5aa16ffe2947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:43.796043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93340d-dc92-40c3-b66d-5aa16ffe2947 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:43.823673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415b3f9-976b-4687-a1d7-73d4b173600d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:43.862850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:43.879094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7307afa-e96b-4193-a4a1-1303bd3401b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:43.880240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307afa-e96b-4193-a4a1-1303bd3401b9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:43.881204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307afa-e96b-4193-a4a1-1303bd3401b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:43.881485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7307afa-e96b-4193-a4a1-1303bd3401b9 None admin] PATH_INFO: `/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:43.883493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7307afa-e96b-4193-a4a1-1303bd3401b9 None admin] RBAC: Authorizing `identity:get_project(project_id=f9592ce6b5e44e00be8e402c2ab7dc65)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:43.886697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7307afa-e96b-4193-a4a1-1303bd3401b9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:43.891371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 85/175] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:43 2022] GET /identity/v3/projects/f9592ce6b5e44e00be8e402c2ab7dc65 => generated 312 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:43.893960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9050599-1727-4fb7-96c9-46ca64a6a32a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:43.894470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9050599-1727-4fb7-96c9-46ca64a6a32a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:43.894953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9050599-1727-4fb7-96c9-46ca64a6a32a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:43.898774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 86/176] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:43.900540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9050599-1727-4fb7-96c9-46ca64a6a32a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:43.928786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 91/177] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:43.930171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271f3f7-d822-42b0-937d-b538be8eb3e5 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:43.930452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271f3f7-d822-42b0-937d-b538be8eb3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:43.930756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271f3f7-d822-42b0-937d-b538be8eb3e5 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:43.932307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 92/178] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:43.933042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271f3f7-d822-42b0-937d-b538be8eb3e5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:44.027757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:44.077707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271f3f7-d822-42b0-937d-b538be8eb3e5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:44.082658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9050599-1727-4fb7-96c9-46ca64a6a32a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:44.119343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:44.120022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.120304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.120579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.121313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] RBAC: Adding query filter params (role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=7e902eff1ce140c9a81aac150e114ae7) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:44.121771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:44.123635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6924bfa-528f-421c-81b5-b808e350408d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:44.130508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 87/179] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:43 2022] GET /identity/v3/role_assignments?user.id=7e902eff1ce140c9a81aac150e114ae7&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:44.131229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8434-3b42-4cbb-8606-c1d831c13a1d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.131512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8434-3b42-4cbb-8606-c1d831c13a1d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.131789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a8434-3b42-4cbb-8606-c1d831c13a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.141432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:44.142270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.142688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.143117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.144083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, scope.project.id=f9592ce6b5e44e00be8e402c2ab7dc65, user.id=7e902eff1ce140c9a81aac150e114ae7) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:44.144552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:44.146241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:44.153430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-142a8434-3b42-4cbb-8606-c1d831c13a1d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:44.154854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 93/180] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:43 2022] GET /identity/v3/role_assignments?user.id=7e902eff1ce140c9a81aac150e114ae7&scope.project.id=f9592ce6b5e44e00be8e402c2ab7dc65&role.id=87804c58f0314370a20ec138295f6863 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:44.155803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:44.164480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-142a8434-3b42-4cbb-8606-c1d831c13a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:44.281563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:44.313401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 88/181] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:44.317123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e6040-57fd-473f-b043-3f666c5dcb05 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.317931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e6040-57fd-473f-b043-3f666c5dcb05 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.319542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e6040-57fd-473f-b043-3f666c5dcb05 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.323148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 89/182] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:44.327130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.328079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.329060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.366591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c062c-56de-47f9-823a-00174aa9590a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:44.370417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:44.373808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:44.440904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d276fc-4ae1-4202-a041-47e4e897219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:44.441968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d276fc-4ae1-4202-a041-47e4e897219f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.442449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d276fc-4ae1-4202-a041-47e4e897219f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.442924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d276fc-4ae1-4202-a041-47e4e897219f None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.444953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d276fc-4ae1-4202-a041-47e4e897219f None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:44.448371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d276fc-4ae1-4202-a041-47e4e897219f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:44.451844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 94/183] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:44 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:44.454489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825a6e-9e5b-4db6-bdc6-214544e7d9ef None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.454489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825a6e-9e5b-4db6-bdc6-214544e7d9ef None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.454926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825a6e-9e5b-4db6-bdc6-214544e7d9ef None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.457420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 95/184] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:44.459430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a499c4-5128-4908-af22-5334bdef23af None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.459952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a499c4-5128-4908-af22-5334bdef23af None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.460430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a499c4-5128-4908-af22-5334bdef23af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.472459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:44.496201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b1a499c4-5128-4908-af22-5334bdef23af None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:44.499911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b1a499c4-5128-4908-af22-5334bdef23af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:44.502252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 90/185] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:44.503011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:44.604177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 96/186] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:44.606042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.607225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.608126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.618385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2226c5-9a17-457c-8b5f-abae04d53f26 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:44.631074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:44.632911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:44.637652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:44.664644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:44.665287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.665629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.666308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.666780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=6fd5eb0c7a1f4bdbb078438d56380575) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:44.667089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:44.668656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:44.676428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 91/187] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:44 2022] GET /identity/v3/role_assignments?user.id=6fd5eb0c7a1f4bdbb078438d56380575&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:44.677408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:44.742854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 97/188] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:44.866520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395cfe5-d91f-4fa2-ad66-9519c266e825 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:44.904909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:44.905103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:44.905699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:44.905985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:44.906265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:44.926143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:44.934089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:44.935080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:44.935995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 92/189] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:44.941713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:44.993158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:45.008646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:45.060214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71623c-d9ae-4dd5-bf3c-6740a4331fd6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.064627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ddc4b-aab2-433e-9b55-6bae95c16ad6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.114346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.115553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.116116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.116650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.117826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:45.118365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:45.121803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:45.134926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.135098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.135324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.135591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.138775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 93/190] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:45.140313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.154243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:45.160772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:45.161723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:45.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 98/191] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:45.163876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.286097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efa8ae-ad7a-4dc3-8ecd-adc138846c38 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.288741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:45.324686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.325803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.326155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.326504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.327708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:45.329221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:45.331044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 94/192] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:45.331844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.351592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae54dd5-6d11-40fd-96e2-38d1e16f0f41 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.379381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:45.393341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.394001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.394263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.394536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] PATH_INFO: `/v3/users/b37b306a34e044c795022a0a658ba47c` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.409711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] RBAC: Authorizing `identity:get_user(user_id=b37b306a34e044c795022a0a658ba47c)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:45.411278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:45.413030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 99/193] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:45 2022] GET /identity/v3/users/b37b306a34e044c795022a0a658ba47c => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:45.413826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.463388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cecd18-0ab5-468d-859e-b2f290299c17 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.509210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.512358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.512358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.512358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.512358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:45.512358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:45.514419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:45.527848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 95/194] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:45.529636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.550590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d212-01f8-4e77-86ff-be148c80aa7f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.568276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:45.592292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.592975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.593252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.593522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.615428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:45.620468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:45.621609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 100/195] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:03:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:45.622953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.660996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e7ac2-1e22-4c0a-ae61-d0d49d8c981b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.679573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:45.700842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.701465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.701755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.702021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.702764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:45.704280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:45.811463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fc9f3-7059-4f73-a5ff-4e07986cb860 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:45.848567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:45.849321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:45.849676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:45.850023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:45.863280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 96/196] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:03:45 2022] POST /identity/v3/users => generated 297 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:45.863280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:45.956663 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:45.959464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.004827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:46.019424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:46.027991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5e02a-2aed-4895-bdd1-777f3492277c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:46.080301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:46.083099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:46.083099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:46.083099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] PATH_INFO: `/v3/users/5f3566fb0a7c451cb5375b1861e939b7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:46.083099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] RBAC: Authorizing `identity:get_user(user_id=5f3566fb0a7c451cb5375b1861e939b7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:46.084559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.086909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 97/197] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:45 2022] GET /identity/v3/users/5f3566fb0a7c451cb5375b1861e939b7 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:46.087554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.135459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 101/198] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:46.136476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.262199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed709b-e839-4cdc-ae0b-47483a37fabb None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:46.324624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f0ff-10cb-4e3d-9c65-e0b9573166ae None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:46.325360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:46.326094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:46.326639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:46.326915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:46.328620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:46.328937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:46.330347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.340302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 98/199] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:46.341683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.374216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:46.412550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:46.413568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:46.414736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:46.415208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:46.438947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:46.487079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a52038-2312-44c4-80c3-178269116da2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:46.538268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:46.542017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.547241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:46.548914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:46.549206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:46.549484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:46.550724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:46.552324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.554532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 99/200] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:46.555541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.709448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:46.723324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 102/201] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:46.724476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.741936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91121e4-ece4-4206-b695-c43b4bb5366e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:46.807048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:46.809137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:46.809871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:46.810186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:46.810710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] PATH_INFO: `/v3/users/b37b306a34e044c795022a0a658ba47c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:46.811741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] RBAC: Authorizing `identity:get_user(user_id=b37b306a34e044c795022a0a658ba47c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:46.813363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.815284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 100/202] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/users/b37b306a34e044c795022a0a658ba47c => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:46.821204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.881916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9725a-ad89-48d5-a0f4-4f11a317e5ab None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:46.928109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:46.929286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:46.929720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:46.930168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:46.931076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=b37b306a34e044c795022a0a658ba47c) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:46.931590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:46.933771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:46.940114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 103/203] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/role_assignments?user.id=b37b306a34e044c795022a0a658ba47c&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:46.941214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:46.971068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8718cf8-3d59-466a-bca0-2b543adfeb60 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:47.021805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:47.022557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:47.022840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:47.023112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:47.024359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:47.025896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:47.028027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 101/204] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:47.036547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:47.115571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcc252-2cd9-471d-bde7-80d9e9588a8e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:47.118487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:47.130341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:47.194070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:47.195236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:47.195740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:47.196272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:47.234773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987e0c5-4037-4626-b738-6b433cbae743 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:47.274741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:47.275520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:47.275811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:47.276131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:47.276876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] RBAC: Adding query filter params (role.id=a84e6592db084ce2b9667f2cd70fdcb2, scope.project.id=263e684247604a1a8e174c46e3574118, user.id=5f3566fb0a7c451cb5375b1861e939b7) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:47.277204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:47.279200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:47.286673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 102/205] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:47 2022] GET /identity/v3/role_assignments?user.id=5f3566fb0a7c451cb5375b1861e939b7&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=a84e6592db084ce2b9667f2cd70fdcb2 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:47.357298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:47.358812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:47.530607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 104/206] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:47.541058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:47.700546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:47.722769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c7d68-bff6-46f6-b75e-9a3ddd9e6b57 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:47.779192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6ae37-ceb1-4e5f-a818-11e2ab27095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:47.779841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae37-ceb1-4e5f-a818-11e2ab27095c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:47.780179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae37-ceb1-4e5f-a818-11e2ab27095c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:47.780592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6ae37-ceb1-4e5f-a818-11e2ab27095c None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:47.782035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae37-ceb1-4e5f-a818-11e2ab27095c None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:47.783491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6ae37-ceb1-4e5f-a818-11e2ab27095c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:47.790065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 103/207] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:47 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:47.791704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:47.796292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:47.959916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e39649-678d-4e04-83d4-f3175225e0c7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:47.999010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:48.000025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:48.000601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:48.001101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:48.002049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] RBAC: Adding query filter params (role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=b37b306a34e044c795022a0a658ba47c) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:48.002555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:48.004265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365aaf57-1ffb-4f2a-9e1d-b71efbe78020 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:48.010883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 105/208] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:47 2022] GET /identity/v3/role_assignments?user.id=b37b306a34e044c795022a0a658ba47c&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:48.513712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e464ee-4158-41a6-a473-a3ef6781887f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:48.514059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e464ee-4158-41a6-a473-a3ef6781887f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:48.514360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e464ee-4158-41a6-a473-a3ef6781887f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:48.515992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 104/209] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:48.522429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ec7b-f7c3-4907-9b81-0609af203bbb None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:48.522912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ec7b-f7c3-4907-9b81-0609af203bbb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:48.523405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ec7b-f7c3-4907-9b81-0609af203bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:48.553862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f4e8ec7b-f7c3-4907-9b81-0609af203bbb None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:48.555776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f4e8ec7b-f7c3-4907-9b81-0609af203bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:48.646822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:48.656431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 106/210] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:48.814595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfd5a1-948a-45ce-9617-50039960eb90 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:48.815262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfd5a1-948a-45ce-9617-50039960eb90 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:48.815548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfd5a1-948a-45ce-9617-50039960eb90 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:48.818343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 105/211] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:48.827543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:48.828125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:48.828592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:48.860566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:48.862641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:48.969394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 107/212] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:49.017093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfd5a1-948a-45ce-9617-50039960eb90 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:49.190525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfd5a1-948a-45ce-9617-50039960eb90 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:49.240104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:49.241130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.241566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.241850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.255280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:49.260302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:49.261351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:49.262798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 106/213] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:49.270080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:49.332342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:49.357013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:49.432764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f093fe4-377d-42d9-901a-b1589e1b5bbf None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:49.461555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1094c29-f2c9-472d-bc09-87339a02b8dc None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:49.466114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:49.476565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:49.477667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.478403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.478942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.480079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:49.480510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:49.482949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:49.497236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 108/214] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:49.504748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:49.513374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:49.515771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.516428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.516951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.518853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:49.520676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:49.522795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 107/215] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:49.533238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:49.644036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:49.660428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd425f8e-c710-41f9-9691-6a7c175c5822 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:49.664882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b1f37-48e8-4022-b4c6-82b4791ebea2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:49.708880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96a93c-8773-4771-9b41-df8792e0ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:49.709448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a93c-8773-4771-9b41-df8792e0ae5a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.709727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a93c-8773-4771-9b41-df8792e0ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.709992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a93c-8773-4771-9b41-df8792e0ae5a None admin] PATH_INFO: `/v3/users/62f6aec0e2ab4f4fbc7134279c755e92` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.722527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f62ec-cbaf-401a-b5ef-146b833229d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:49.723153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f62ec-cbaf-401a-b5ef-146b833229d4 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.723427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f62ec-cbaf-401a-b5ef-146b833229d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.723775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f62ec-cbaf-401a-b5ef-146b833229d4 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.724475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f62ec-cbaf-401a-b5ef-146b833229d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:49.726071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f62ec-cbaf-401a-b5ef-146b833229d4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:49.732822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96a93c-8773-4771-9b41-df8792e0ae5a None admin] RBAC: Authorizing `identity:get_user(user_id=62f6aec0e2ab4f4fbc7134279c755e92)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:49.735196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96a93c-8773-4771-9b41-df8792e0ae5a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:49.742626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 109/216] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:49 2022] GET /identity/v3/users/62f6aec0e2ab4f4fbc7134279c755e92 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:49.742626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416fdc2-54e2-4c89-accb-0732642b8b99 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.742626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416fdc2-54e2-4c89-accb-0732642b8b99 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.742626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416fdc2-54e2-4c89-accb-0732642b8b99 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.746967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 110/217] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:49.746967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416fdc2-54e2-4c89-accb-0732642b8b99 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:49.805786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:49.855153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 108/218] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:03:49 2022] POST /identity/v3/users => generated 297 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:03:49.857399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.858045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.858622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:49.890369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:49.892275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:49.893385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416fdc2-54e2-4c89-accb-0732642b8b99 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:49.957074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab54f3-0592-42c9-b10f-8638af8e96e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:49.957711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab54f3-0592-42c9-b10f-8638af8e96e8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:49.957992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab54f3-0592-42c9-b10f-8638af8e96e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:49.958314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab54f3-0592-42c9-b10f-8638af8e96e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:50.030766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 109/219] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:50.080244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab54f3-0592-42c9-b10f-8638af8e96e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:50.081691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab54f3-0592-42c9-b10f-8638af8e96e8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:50.150670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:50.219219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 111/220] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:50.229165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:50.335152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e55b-7a99-4ffa-8857-dec0638a9693 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:50.335494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e55b-7a99-4ffa-8857-dec0638a9693 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:50.335718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e55b-7a99-4ffa-8857-dec0638a9693 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:50.338696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 112/221] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:50.343335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:50.343623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:50.343971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:50.371355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:50.373831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:50.410083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:50.430990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d88a27-99d8-4ef8-b037-ae98cc719fcf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:50.491217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 113/222] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:03:50.492253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:50.492505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:50.493521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:50.494044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:50.494556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:50.497465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:50.500645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:50.504198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 110/223] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:50 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:50.513832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:50.694391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:50.737532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85abaed-fb58-469b-a258-0976bf9ccc32 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:50.791011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040f849-39ac-4c70-b481-dbe8c40a90d4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:50.815603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:50.817038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:50.818882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:50.819455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:50.850341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:50.858389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:50.859411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:50.860529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:50.863278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 114/224] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:50.872944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:50.879962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:50.880598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:50.880862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:50.881137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:50.883337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=62f6aec0e2ab4f4fbc7134279c755e92) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:50.883865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:50.886483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:50.915085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 111/225] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:50 2022] GET /identity/v3/role_assignments?user.id=62f6aec0e2ab4f4fbc7134279c755e92&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=87804c58f0314370a20ec138295f6863 => generated 259 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:50.934999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:50.938471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:51.055935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd6563-914e-4bda-bc84-67c78c0647df None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.111175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.112029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.114025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.114564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.115709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:51.117514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:51.119910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:51.128083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:51.145387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 115/226] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:51.150411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02344b96-80aa-433f-be56-ae97590ce626 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.154135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:51.218450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.219172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.219453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.219729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.234000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:51.241020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:51.243808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:51.246164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 112/227] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:03:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:51.252045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:51.328377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:51.376106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836178cc-b312-475f-ada9-7e6655a7232d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.425242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.426606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.427318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.427829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.429229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:51.429765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:51.431859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:51.447039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 113/228] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:03:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:51.457379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:51.465473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120109c-37b5-43e9-92a4-a19fa58a8065 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.603171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9b460-872e-475a-8c2f-190ac5c74f2f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.624915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.626082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.626295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.626702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] PATH_INFO: `/v3/users/b37b306a34e044c795022a0a658ba47c` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.629407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] RBAC: Authorizing `identity:get_user(user_id=b37b306a34e044c795022a0a658ba47c)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:51.632215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:51.634674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 116/229] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:51 2022] GET /identity/v3/users/b37b306a34e044c795022a0a658ba47c => generated 283 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:51.641616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:51.645013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:51.648098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.649011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.653054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.653054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] PATH_INFO: `/v3/users/b37b306a34e044c795022a0a658ba47c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.653054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] RBAC: Authorizing `identity:get_user(user_id=b37b306a34e044c795022a0a658ba47c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:51.653054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:51.654784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 114/230] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:51 2022] GET /identity/v3/users/b37b306a34e044c795022a0a658ba47c => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:51.674767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:51.749142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:51.798165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0764c-4254-4ec0-8cb3-2b0984e8ff8c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.860160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.861232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.861676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.862105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.863458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:51.865184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:51.867021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 117/231] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:51 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:51.868124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024b1b6-2230-4416-b608-322e0a94a8b0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:51.878078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:51.929317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:51.930819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:51.931256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:51.931704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:51.933403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:51.935222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:51.937755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 115/232] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:51 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:51.945914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:52.092657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:52.128064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e996d-40e4-43a6-bb37-a38df1d5ec27 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:52.181116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfbd94-5685-418d-921a-62aa46cb5bfd None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:52.223875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af08c98-3f50-45fa-be87-6f1645e5e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:52.224729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08c98-3f50-45fa-be87-6f1645e5e938 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:52.225012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08c98-3f50-45fa-be87-6f1645e5e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:52.225286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af08c98-3f50-45fa-be87-6f1645e5e938 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/b37b306a34e044c795022a0a658ba47c/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:52.228342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af08c98-3f50-45fa-be87-6f1645e5e938 None admin] RBAC: Authorizing `identity:create_grant(project_id=23412b7c545d48c5b5fce56d6eefb33e, user_id=b37b306a34e044c795022a0a658ba47c, role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:52.231627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af08c98-3f50-45fa-be87-6f1645e5e938 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:52.250663 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804aaf5-a4c7-4aaf-b206-30b93ed13be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:52.251724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804aaf5-a4c7-4aaf-b206-30b93ed13be6 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:52.252206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804aaf5-a4c7-4aaf-b206-30b93ed13be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:52.252591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804aaf5-a4c7-4aaf-b206-30b93ed13be6 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/b37b306a34e044c795022a0a658ba47c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:52.256085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804aaf5-a4c7-4aaf-b206-30b93ed13be6 None admin] RBAC: Authorizing `identity:create_grant(project_id=23412b7c545d48c5b5fce56d6eefb33e, user_id=b37b306a34e044c795022a0a658ba47c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:52.259027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804aaf5-a4c7-4aaf-b206-30b93ed13be6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:52.265074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:52.335248 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 116/233] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:51 2022] PUT /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/b37b306a34e044c795022a0a658ba47c/roles/c2a3834a436b40deabcb87f412ca5322 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:03:52.375560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 118/234] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:51 2022] PUT /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/b37b306a34e044c795022a0a658ba47c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:03:52.873542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e42041-91c1-4743-86e2-a3c5e8bf657c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:52.874230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e42041-91c1-4743-86e2-a3c5e8bf657c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:52.874763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e42041-91c1-4743-86e2-a3c5e8bf657c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:52.876406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 117/235] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:52.882074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:52.882613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:52.883088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:52.903399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:52.905022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:52.912390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd1a1d-a284-497d-8d96-1c5206620ebf None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:52.912690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd1a1d-a284-497d-8d96-1c5206620ebf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:52.912965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd1a1d-a284-497d-8d96-1c5206620ebf None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:52.914612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 118/236] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:52.923106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:52.923497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:52.923811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:52.969523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:52.971403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:53.014569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 119/237] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:53.131504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 119/238] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:53.604243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:53.756077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:53.823347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f01c81-7239-4aca-bc69-5dfdc541162b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:53.876391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:53.879622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:53.904059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:53.904870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:53.905356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:53.905842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:53.907858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:53.910889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:53.913976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 120/239] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:53.922757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:53.982230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03b0af-6dcc-47e5-b708-fff172816e5b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:54.030452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a20934-8568-4b76-b21e-dd74e01cc26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:54.031522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20934-8568-4b76-b21e-dd74e01cc26b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.031811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20934-8568-4b76-b21e-dd74e01cc26b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.032142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a20934-8568-4b76-b21e-dd74e01cc26b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.063209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a20934-8568-4b76-b21e-dd74e01cc26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-d2a20934-8568-4b76-b21e-dd74e01cc26b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:54.073799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:54.075937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:03:54.077293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:54.085404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 120/240] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:54.091121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1df5-e0df-44a6-890c-886ef0a40a8a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.092041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1df5-e0df-44a6-890c-886ef0a40a8a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.092798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1df5-e0df-44a6-890c-886ef0a40a8a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.095874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 121/241] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:03:54.097660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1df5-e0df-44a6-890c-886ef0a40a8a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:54.193678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c306e-02fb-4578-ab85-7e55572d14c0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:54.256712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cbe8c-7144-4c84-82cc-7500c855c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:54.257340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cbe8c-7144-4c84-82cc-7500c855c07d None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.257829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cbe8c-7144-4c84-82cc-7500c855c07d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.258284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cbe8c-7144-4c84-82cc-7500c855c07d None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.259412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cbe8c-7144-4c84-82cc-7500c855c07d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:54.262166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cbe8c-7144-4c84-82cc-7500c855c07d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:54.288529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1df5-e0df-44a6-890c-886ef0a40a8a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:54.302603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:54.339626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:54.341801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.343025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.344354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.345774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:54.346601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:54.349533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:54.356045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 121/242] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:03:53 2022] POST /identity/v3/users => generated 293 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:03:54.358672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.359190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.359878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.370100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 122/243] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:54.380747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:54.395420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:54.396812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:54.476278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:54.523119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 122/244] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:54.577907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90be4f7-c087-4897-9140-4367164860f4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:54.627519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:54.628551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.629039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.629644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] PATH_INFO: `/v3/users/3c5bd10f9d7643b8880f343cad8bdea8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.650811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] RBAC: Authorizing `identity:get_user(user_id=3c5bd10f9d7643b8880f343cad8bdea8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:54.653892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:54.656005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 123/245] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:54 2022] GET /identity/v3/users/3c5bd10f9d7643b8880f343cad8bdea8 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:54.662343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:54.833004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f40bf-eb91-4aaa-b0bf-981cde62be36 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:54.901648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:54.903288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:54.903867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:54.904513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:54.922688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:54.974563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:54.985869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:55.017197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:55.019343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:55.078700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d5c3-51f2-450d-87b4-13c52196d569 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:55.125504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:55.126440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.126872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.127416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.149372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:55.159414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:55.160352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:55.161196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 124/246] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:55.176048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:55.228094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 123/247] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:03:55.231290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:55.326384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:55.346335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3665b-823f-4c7e-91bb-4f52e18c0fe3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:55.381484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:55.413540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923409fe-ae64-4e31-a063-5fec33516070 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:55.436044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:55.437822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.438303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.438773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.440333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:55.441191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:55.444565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7deca-69a0-45e0-8f5a-328258d2b2ce None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:55.464165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 125/248] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:55.468001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38300ee3-f49a-495e-beaf-f6a6c683b7ae None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.468626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38300ee3-f49a-495e-beaf-f6a6c683b7ae None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.469183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38300ee3-f49a-495e-beaf-f6a6c683b7ae None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.472239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 126/249] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:55.478980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38300ee3-f49a-495e-beaf-f6a6c683b7ae None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:55.485311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26721dc5-136a-46b5-b127-e6bfe393c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:55.486367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721dc5-136a-46b5-b127-e6bfe393c551 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.486919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721dc5-136a-46b5-b127-e6bfe393c551 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.487459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26721dc5-136a-46b5-b127-e6bfe393c551 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.491074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721dc5-136a-46b5-b127-e6bfe393c551 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:55.494306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26721dc5-136a-46b5-b127-e6bfe393c551 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:55.498735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 124/250] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:55 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:55.506286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.506805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.507348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.544727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:55.546468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:55.667449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38300ee3-f49a-495e-beaf-f6a6c683b7ae None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:55.670710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 125/251] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:55.675926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:55.774090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17740e7f-0705-432a-bab0-a867abbc12ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:55.775643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17740e7f-0705-432a-bab0-a867abbc12ad None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.776492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17740e7f-0705-432a-bab0-a867abbc12ad None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.777277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17740e7f-0705-432a-bab0-a867abbc12ad None admin] PATH_INFO: `/v3/users/62f6aec0e2ab4f4fbc7134279c755e92` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.779642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17740e7f-0705-432a-bab0-a867abbc12ad None admin] RBAC: Authorizing `identity:get_user(user_id=62f6aec0e2ab4f4fbc7134279c755e92)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:55.782768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17740e7f-0705-432a-bab0-a867abbc12ad None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:55.786276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 127/252] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:55 2022] GET /identity/v3/users/62f6aec0e2ab4f4fbc7134279c755e92 => generated 297 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:55.788724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b852-e7d6-49a2-b210-f8cdba9e8e15 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:55.789893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b852-e7d6-49a2-b210-f8cdba9e8e15 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:55.791949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b852-e7d6-49a2-b210-f8cdba9e8e15 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:55.795919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 128/253] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:55.799480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4b852-e7d6-49a2-b210-f8cdba9e8e15 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:55.852533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:55.862155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:55.923560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1fec5-46e4-4b4c-bed3-04feded5e8fa None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:56.028637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:56.030156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:56.030888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:56.031637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:56.033223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] RBAC: Adding query filter params (role.id=a84e6592db084ce2b9667f2cd70fdcb2, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=3c5bd10f9d7643b8880f343cad8bdea8) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:56.034021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:56.036919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4dcd3e-05bc-44b0-8106-44bc3f8ce8a7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:56.052210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 126/254] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:55 2022] GET /identity/v3/role_assignments?user.id=3c5bd10f9d7643b8880f343cad8bdea8&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=a84e6592db084ce2b9667f2cd70fdcb2 => generated 259 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:56.056003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:56.057157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:56.059930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:56.095798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4b852-e7d6-49a2-b210-f8cdba9e8e15 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:56.109799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:56.115934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:56.254296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:56.255266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:56.255743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:56.256244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:56.258369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:56.265855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:56.267713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 129/255] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:55 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:56.270452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 127/256] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:56.270760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:56.277782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:56.506545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:56.529207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fda96-6f46-47b9-8c46-d5cf334a5a73 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:56.563796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:56.613401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:56.614316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:56.614597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:56.614964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:56.653556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:56.659842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:56.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:03:56.662935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 130/257] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:03:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:56.669831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4943-e1a3-4d0d-9755-d73a120e9fec None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:56.673925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:56.783341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:56.784298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:56.784741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:56.785185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/62f6aec0e2ab4f4fbc7134279c755e92/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:56.794176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] RBAC: Authorizing `identity:create_grant(project_id=23412b7c545d48c5b5fce56d6eefb33e, user_id=62f6aec0e2ab4f4fbc7134279c755e92, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:56.796155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:56.890507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 128/258] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:03:56 2022] PUT /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/62f6aec0e2ab4f4fbc7134279c755e92/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:03:56.899402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:57.005030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95654048-9211-443e-81bd-8bdb85e0fff5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:57.165062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:57.167167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:57.168386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:57.169002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:57.170304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:57.170964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:57.173642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:57.191561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa310d7d-4493-4420-bbfd-8bda83e25338 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:57.203783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 131/259] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 530 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:57.220461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:57.288377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:57.316680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:57.330024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:57.331072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:57.331542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:57.334129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:57.382347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:57.392633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:57.394342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:03:57.395963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:57.402994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 129/260] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:03:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 503 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:03:57.421863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:57.530961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788e05-df16-46b9-b6f0-9e5de24496e9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:57.612731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:57.613454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:57.613734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:57.614009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] PATH_INFO: `/v3/users/b37b306a34e044c795022a0a658ba47c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:57.620610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] RBAC: Authorizing `identity:get_user(user_id=b37b306a34e044c795022a0a658ba47c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:57.625114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:57.626822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 132/261] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:57 2022] GET /identity/v3/users/b37b306a34e044c795022a0a658ba47c => generated 283 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:03:57.635645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:57.829853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34d6bb-887c-4d9e-9eca-321e71f856dd None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:57.841165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:57.903672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385512d-a716-42d2-96c7-0074a7fe55b2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:57.917032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:57.918570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:57.919067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:57.919553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:57.920801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:57.921328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:57.923834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:57.943690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 130/262] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:03:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:57.952572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:58.013746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f80e3-dd5d-4bfb-8a3c-62f45b7edd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:58.014748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f80e3-dd5d-4bfb-8a3c-62f45b7edd07 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:58.015208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f80e3-dd5d-4bfb-8a3c-62f45b7edd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:58.015670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f80e3-dd5d-4bfb-8a3c-62f45b7edd07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:58.106227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:58.230475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f80e3-dd5d-4bfb-8a3c-62f45b7edd07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:58.240902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f80e3-dd5d-4bfb-8a3c-62f45b7edd07 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:58.249370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa0d44-5f9c-4085-800c-af304c6df984 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:58.352811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:58.353450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:58.353734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:58.354010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] PATH_INFO: `/v3/users/b37b306a34e044c795022a0a658ba47c` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:58.357350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] RBAC: Authorizing `identity:get_user(user_id=b37b306a34e044c795022a0a658ba47c)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:58.358975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:58.362617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 131/263] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:57 2022] GET /identity/v3/users/b37b306a34e044c795022a0a658ba47c => generated 283 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:58.368754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:58.451205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:58.475676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 133/264] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:58.477470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7f981-adc2-4a52-87b5-bd68d2d1353e None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:58.477820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7f981-adc2-4a52-87b5-bd68d2d1353e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:58.478303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7f981-adc2-4a52-87b5-bd68d2d1353e None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:58.480513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 134/265] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:58.486177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7f981-adc2-4a52-87b5-bd68d2d1353e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:58.614536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7f981-adc2-4a52-87b5-bd68d2d1353e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:58.653816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d5d75-101f-4a65-9658-92eee8809e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:58.654872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d5d75-101f-4a65-9658-92eee8809e61 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:58.655352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d5d75-101f-4a65-9658-92eee8809e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:58.655769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d5d75-101f-4a65-9658-92eee8809e61 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:58.657805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d5d75-101f-4a65-9658-92eee8809e61 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:58.659420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d5d75-101f-4a65-9658-92eee8809e61 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:58.662001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 135/266] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:58 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:58.663389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:58.663818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:58.668756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:58.682920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d5b68-0cb6-450d-acb4-e4d49914868a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:58.686242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:58.687567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:58.760322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:58.772325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:58.788763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:58.789400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:58.789685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:58.789964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:58.796029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 136/267] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:58.797223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:58.928091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:58.930530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:58.977005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd2aab-7572-45de-91e8-416f4bad7469 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:59.017715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:59.018335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.018784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.019229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.020264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=b37b306a34e044c795022a0a658ba47c) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:59.020719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:59.022437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e9d20-98e8-4988-9abf-65343fd89df6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:59.029642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 137/268] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:58 2022] GET /identity/v3/role_assignments?user.id=b37b306a34e044c795022a0a658ba47c&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:59.031251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558fe0b-68b9-4a54-bc0d-a5244a6fca3f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.031695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558fe0b-68b9-4a54-bc0d-a5244a6fca3f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.032152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558fe0b-68b9-4a54-bc0d-a5244a6fca3f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.034161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 138/269] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:59.042323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.042901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.043320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.082665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:59.084659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:59.182917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:59.187542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 132/270] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:03:59.191095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:59.286768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:59.292997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 139/271] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:03:59.294003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:59.362531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd4926-45cf-4420-83fc-8933ed3eea1f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:59.427415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b973f0d-46c1-4dd0-b759-7d1fa83ab0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:59.428223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b973f0d-46c1-4dd0-b759-7d1fa83ab0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.428506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b973f0d-46c1-4dd0-b759-7d1fa83ab0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.428778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b973f0d-46c1-4dd0-b759-7d1fa83ab0c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.458641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b973f0d-46c1-4dd0-b759-7d1fa83ab0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7b973f0d-46c1-4dd0-b759-7d1fa83ab0c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:03:59.466970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:03:59.467928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:03:59.469164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 133/272] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:03:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:03:59.469828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4bf65-4c22-4592-abed-a96fd2e743e3 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.470192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4bf65-4c22-4592-abed-a96fd2e743e3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.470347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4bf65-4c22-4592-abed-a96fd2e743e3 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.476765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 134/273] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:03:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:03:59.483259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.483548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.483829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.492217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf797c-2cc9-4851-87c3-4824aa21b95a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:59.505520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:03:59.507368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:03:59.552895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:59.553794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.554237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.554623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.556321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:59.559594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:59.563013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 140/274] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:03:59 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:59.568126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:59.603578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 135/275] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 21:03:59.604730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:59.636467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:59.652694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:03:59.765804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be55d5f7-afd0-49a1-9628-84dfd29efd68 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:59.794849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d050f3-b771-4c8c-8ebd-a3f78b468010 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:03:59.826730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:59.827771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.828337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.829704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.831319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:59.831864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:59.835659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:59.847702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:03:59.848573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:03:59.848967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:03:59.849361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:03:59.850205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] RBAC: Adding query filter params (role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=b37b306a34e044c795022a0a658ba47c) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:03:59.850624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:03:59.852219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:03:59.854508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 141/276] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:03:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:59.857422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:03:59.860638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 136/277] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:03:59 2022] GET /identity/v3/role_assignments?user.id=b37b306a34e044c795022a0a658ba47c&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:03:59.865272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:00.056799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:00.059633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:00.064527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961ed1f-7671-443f-a582-b64ee4333ce1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:00.096526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02433dce-890c-49f7-aac9-bfc203855609 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:00.123164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:00.129608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.129608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.129796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.131538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:00.133153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:00.135194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 142/278] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:03:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:00.135719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:00.163567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:00.164608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.165101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.165601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] PATH_INFO: `/v3/users/a9936b18c5834a5a9088df1fc46dc79b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.184372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] RBAC: Authorizing `identity:get_user(user_id=a9936b18c5834a5a9088df1fc46dc79b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:00.186984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:00.189357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 137/279] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:03:59 2022] GET /identity/v3/users/a9936b18c5834a5a9088df1fc46dc79b => generated 293 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:00.190396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:00.289956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07add5e6-d8c2-4c0e-b4b5-5d9732fcb7f3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:00.323419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c7f06-1773-439f-95fe-434de80bcc16 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:00.326206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:00.326857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.327139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.327431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.339210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:00.344733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:00.348399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 143/280] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:00.349206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:00.381577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38544573-7f5c-4f77-824c-a9e40f704eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:00.382357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544573-7f5c-4f77-824c-a9e40f704eb0 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.382638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544573-7f5c-4f77-824c-a9e40f704eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.382913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38544573-7f5c-4f77-824c-a9e40f704eb0 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.383576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544573-7f5c-4f77-824c-a9e40f704eb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:00.385089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38544573-7f5c-4f77-824c-a9e40f704eb0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:00.388592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:00.393867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:00.476099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652fbab-a314-400d-a356-a11591a4f6c5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:00.491940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 138/281] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:04:00 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:04:00.493569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7b3d3-07e1-413d-9804-46bc06a40d88 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.493863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7b3d3-07e1-413d-9804-46bc06a40d88 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.494141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7b3d3-07e1-413d-9804-46bc06a40d88 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.495926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 139/282] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:00.496790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7b3d3-07e1-413d-9804-46bc06a40d88 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:00.544295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:00.545062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.545344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.545620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.546453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, user.id=a9936b18c5834a5a9088df1fc46dc79b, scope.system=all) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:00.546763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:00.548373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ff6d6-9617-49e6-82ed-20522750a738 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:00.555150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 144/283] 10.222.0.65 () {56 vars in 1308 bytes} [Mon Aug 29 21:04:00 2022] GET /identity/v3/role_assignments?user.id=a9936b18c5834a5a9088df1fc46dc79b&scope.system=all&role.id=87804c58f0314370a20ec138295f6863 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:00.568236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.568236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.568236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.590921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:00.592260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:00.678689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7b3d3-07e1-413d-9804-46bc06a40d88 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:00.712848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 145/284] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:00.748387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:00.749029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:00.749432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:00.749714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:00.750401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:00.750709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:00.752237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:00.766154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 140/285] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:00.772228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:00.819626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:00.952410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43b634-d10b-46e6-a39b-d19d9a9b4377 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:01.024015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:01.024980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:01.025464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:01.025919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] PATH_INFO: `/v3/users/3c5bd10f9d7643b8880f343cad8bdea8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:01.028391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] RBAC: Authorizing `identity:get_user(user_id=3c5bd10f9d7643b8880f343cad8bdea8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:01.030106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:01.032106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 146/286] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:00 2022] GET /identity/v3/users/3c5bd10f9d7643b8880f343cad8bdea8 => generated 297 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:01.040236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:01.054319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:01.152565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:01.218000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7caa5-fdef-44e4-a208-b542feaa2043 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:01.293934 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:01.311866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:01.312807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:01.313210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:01.313609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:01.316924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:01.319169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:01.321927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 141/287] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:01 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:01.330886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609fcb-7415-4968-aa77-f2574b3c7214 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:01.350010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:01.408612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:01.413302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:01.413302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:01.413302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:01.433798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:01.446183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:01.447745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:01.450308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:01.452906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 147/288] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:01.457766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:01.610102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83355428-40b4-4716-9691-b6eeb912406a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:01.666296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:01.675916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:01.675916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:01.675916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:01.675916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:01.675916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:01.675916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:01.687712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 148/289] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:01.696936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:01.715040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7a564-9860-452c-b7c9-01218dbb6059 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:01.740340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:01.746937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:01.777480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:01.778234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:01.778628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:01.778782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/3c5bd10f9d7643b8880f343cad8bdea8/roles/a84e6592db084ce2b9667f2cd70fdcb2` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:01.784176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] RBAC: Authorizing `identity:create_grant(project_id=23412b7c545d48c5b5fce56d6eefb33e, user_id=3c5bd10f9d7643b8880f343cad8bdea8, role_id=a84e6592db084ce2b9667f2cd70fdcb2)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:01.785743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:01.852153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 142/290] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:04:01 2022] PUT /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/users/3c5bd10f9d7643b8880f343cad8bdea8/roles/a84e6592db084ce2b9667f2cd70fdcb2 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:01.885857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc9cf9-e91a-4121-bc0a-3685dc934cee None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:01.925134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:01.925791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:01.926073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:01.926345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] PATH_INFO: `/v3/users/62f6aec0e2ab4f4fbc7134279c755e92` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:01.927602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] RBAC: Authorizing `identity:get_user(user_id=62f6aec0e2ab4f4fbc7134279c755e92)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:01.929195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:01.930935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 149/291] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:01 2022] GET /identity/v3/users/62f6aec0e2ab4f4fbc7134279c755e92 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:01.938099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:02.151348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815d108-02d4-4e4c-834b-455649a11d9e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:02.209528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:02.210515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:02.211033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:02.211549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:02.298080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:02.321734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:02.323341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:02.455838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 143/292] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:04:02.465426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:02.659858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:02.701992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126cce55-5532-4b0f-b883-017bb6b55fff None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:02.772192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac7a76-c39d-43c9-9ed4-755f509ca1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:02.772740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac7a76-c39d-43c9-9ed4-755f509ca1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:02.773025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac7a76-c39d-43c9-9ed4-755f509ca1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:02.773305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac7a76-c39d-43c9-9ed4-755f509ca1e9 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:02.777720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac7a76-c39d-43c9-9ed4-755f509ca1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:02.779209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac7a76-c39d-43c9-9ed4-755f509ca1e9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:02.781264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 150/293] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:02 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:02.788053 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:02.971755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:03.011914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02634be9-43d8-4df3-b505-ec89ba1ccac0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:03.092166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:03.092823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.093131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.093468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.094248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=62f6aec0e2ab4f4fbc7134279c755e92) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:03.094586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:03.096115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064877ce-d307-4b23-9635-b8eb81bda3d5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:03.102974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 144/294] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:02 2022] GET /identity/v3/role_assignments?user.id=62f6aec0e2ab4f4fbc7134279c755e92&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=87804c58f0314370a20ec138295f6863 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:03.387712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c28331-32b1-4a60-ac1f-549d18d18e95 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.387998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c28331-32b1-4a60-ac1f-549d18d18e95 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.388228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c28331-32b1-4a60-ac1f-549d18d18e95 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.389883 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 151/295] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:03.398923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dc4aa-0636-4871-a759-c8772ecffbc5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.399774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dc4aa-0636-4871-a759-c8772ecffbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.399774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552dc4aa-0636-4871-a759-c8772ecffbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.423017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-552dc4aa-0636-4871-a759-c8772ecffbc5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:03.424935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-552dc4aa-0636-4871-a759-c8772ecffbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:03.502283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba9c8-c478-4851-890f-30931c7bdb37 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.505673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba9c8-c478-4851-890f-30931c7bdb37 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.505673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba9c8-c478-4851-890f-30931c7bdb37 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.505673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 152/296] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:04:03.515298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76f4a0-c069-44ff-a69b-c4df918c82b9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.515676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76f4a0-c069-44ff-a69b-c4df918c82b9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.516043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76f4a0-c069-44ff-a69b-c4df918c82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.545626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6c76f4a0-c069-44ff-a69b-c4df918c82b9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:03.547535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6c76f4a0-c069-44ff-a69b-c4df918c82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:03.547783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 145/297] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:03.627942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 153/298] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:03.668334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640be896-993b-4f34-afe7-c9f07d99ba00 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.668716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640be896-993b-4f34-afe7-c9f07d99ba00 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.669143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640be896-993b-4f34-afe7-c9f07d99ba00 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.673458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 146/299] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:03.680535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:03.681004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:03.681483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:03.711211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:03.713069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:03.821704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 154/300] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:03.862473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640be896-993b-4f34-afe7-c9f07d99ba00 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:03.949284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:03.960375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:03.982996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:04.023400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640be896-993b-4f34-afe7-c9f07d99ba00 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.082543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.083399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.083780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.084088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.099001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:04.106543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:04.107662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:04.109175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 147/301] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:04.114159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:04.117387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1faefb-1aa1-4f3d-8899-1dbba55df373 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.161721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.162794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.163335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.163920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.165754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:04.167636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:04.169798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 155/302] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:04.176094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:04.283683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f4c57-ed34-4479-a0f1-d87e2f8efe4d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.289123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:04.321216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a409e-addb-4778-9e2d-951f86a6f448 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.344981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.345856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.346254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.347180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.348723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:04.349221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:04.351569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:04.355103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:04.369613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 148/303] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:04.375724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:04.387800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.388539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.388864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.389141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] PATH_INFO: `/v3/groups` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.389803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:04.391333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:04.471809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 156/304] 10.222.0.65 () {60 vars in 1134 bytes} [Mon Aug 29 21:04:04 2022] POST /identity/v3/groups => generated 229 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:04:04.473261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:04.541839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ef934-19f6-4c3a-960f-b8f4e872a24f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.605693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.606961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.607404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.607838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.624449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:04.629055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c48cf2-d904-477c-a096-1d4d4f4973d4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:04.633664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:04.634974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:04.637224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:04.639577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 149/305] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:04.649655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:04.687546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.688312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.688600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.689392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] PATH_INFO: `/v3/users/a9936b18c5834a5a9088df1fc46dc79b` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.693380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] RBAC: Authorizing `identity:get_user(user_id=a9936b18c5834a5a9088df1fc46dc79b)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:04.693380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:04.694555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 157/306] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:04 2022] GET /identity/v3/users/a9936b18c5834a5a9088df1fc46dc79b => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:04.701147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:04.706400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:04.763147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:04.832260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450e8dc-7bb7-4c33-8823-17bc5facfe54 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.851645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b85a0b-a866-4e4f-8001-10f1c4c7c2bf None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:04.893578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc11a5e-5470-4b00-9818-43aa7bb08814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.894566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc11a5e-5470-4b00-9818-43aa7bb08814 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.894810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc11a5e-5470-4b00-9818-43aa7bb08814 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.895206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc11a5e-5470-4b00-9818-43aa7bb08814 None admin] PATH_INFO: `/v3/system/users/a9936b18c5834a5a9088df1fc46dc79b/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.896875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc11a5e-5470-4b00-9818-43aa7bb08814 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a9936b18c5834a5a9088df1fc46dc79b, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:04.898510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc11a5e-5470-4b00-9818-43aa7bb08814 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:04.915319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:04.918279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:04.918279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:04.918279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:04.918684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:04.919236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:04.921946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:04.934601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 150/307] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:04.940460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:05.006093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 158/308] 10.222.0.65 () {58 vars in 1334 bytes} [Mon Aug 29 21:04:04 2022] PUT /identity/v3/system/users/a9936b18c5834a5a9088df1fc46dc79b/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:04:05.081712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c203c3-805e-4ea5-a10f-31337de83e0f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:05.082198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c203c3-805e-4ea5-a10f-31337de83e0f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:05.082660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c203c3-805e-4ea5-a10f-31337de83e0f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:05.092449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 159/309] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:05 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:04:05.096867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:05.097294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:05.097727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:05.116833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af2b92-41f8-4d58-9289-fc7b60279a95 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:05.130805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:05.133672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:05.158578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:05.195918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:05.197584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:05.198655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:05.199189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] PATH_INFO: `/v3/users/80784962154443a88dbde1e0e0560359` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:05.218797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] RBAC: Authorizing `identity:get_user(user_id=80784962154443a88dbde1e0e0560359)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:05.220332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:05.222395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 151/310] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:04 2022] GET /identity/v3/users/80784962154443a88dbde1e0e0560359 => generated 293 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:05.233645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:05.265617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:05.277973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 160/311] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:05.386560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dde37-65b2-4cd5-8de1-ef92a29751e5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:05.432033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:05.433922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:05.434610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:05.435278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:05.436930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] RBAC: Adding query filter params (role.id=a84e6592db084ce2b9667f2cd70fdcb2, user.id=80784962154443a88dbde1e0e0560359, scope.system=all) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:05.437705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:05.440505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:05.449423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 152/312] 10.222.0.65 () {56 vars in 1308 bytes} [Mon Aug 29 21:04:05 2022] GET /identity/v3/role_assignments?user.id=80784962154443a88dbde1e0e0560359&scope.system=all&role.id=a84e6592db084ce2b9667f2cd70fdcb2 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:05.619741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:05.764968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a14d85-cfd2-4fea-8f49-e217bb2ce948 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:05.821506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:05.822471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:05.823021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:05.823399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:05.840160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:05.848457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:05.850032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:05.851479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 161/313] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:05.858993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:05.925153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:06.014590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49172ce1-fd75-4fb1-8074-8b95a03f2cc6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:06.076073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:06.077348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:06.078068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:06.078740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:06.079686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:06.080293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:06.081979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:06.092665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 153/314] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:06.104201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:06.174958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:06.223414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92541c37-60ee-46f9-aea8-ed3c2985aa7c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:06.269509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:06.270509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:06.270959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:06.271412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] PATH_INFO: `/v3/users/3c5bd10f9d7643b8880f343cad8bdea8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:06.273307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] RBAC: Authorizing `identity:get_user(user_id=3c5bd10f9d7643b8880f343cad8bdea8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:06.275381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:06.278155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 162/315] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:06 2022] GET /identity/v3/users/3c5bd10f9d7643b8880f343cad8bdea8 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:06.284974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:06.427843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35330c0f-9385-48da-b8eb-08319d91db5d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:06.429607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:06.471805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:06.472499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:06.472785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:06.473069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:06.547039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:06.548889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:06.659675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 154/316] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:04:06.666060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:06.831627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b71d2-ee5a-4295-9c91-9b8a7d4f1abc None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:06.867265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:06.878196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb64bd-c473-4e51-a237-d5aa33d73e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:06.880498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb64bd-c473-4e51-a237-d5aa33d73e4f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:06.880498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb64bd-c473-4e51-a237-d5aa33d73e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:06.880498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb64bd-c473-4e51-a237-d5aa33d73e4f None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:06.880744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb64bd-c473-4e51-a237-d5aa33d73e4f None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:06.882519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb64bd-c473-4e51-a237-d5aa33d73e4f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:06.884773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 163/317] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:06 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:06.892612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:06.897006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:07.034847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a03cd-185f-4390-96f7-48b1556a1270 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:07.077153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:07.078590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.079358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.080164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.081909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] RBAC: Adding query filter params (role.id=a84e6592db084ce2b9667f2cd70fdcb2, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e, user.id=3c5bd10f9d7643b8880f343cad8bdea8) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:07.082785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:07.085868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90b164-f39a-4ceb-8d62-5edf73d53c8a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:07.098828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 155/318] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:06 2022] GET /identity/v3/role_assignments?user.id=3c5bd10f9d7643b8880f343cad8bdea8&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=a84e6592db084ce2b9667f2cd70fdcb2 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:07.154519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae634f-f421-424e-9a7a-1926f44f0415 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.155482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae634f-f421-424e-9a7a-1926f44f0415 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.156338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae634f-f421-424e-9a7a-1926f44f0415 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.159296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 164/319] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:07.165287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274781-22c2-441f-9d2f-74e0bf60823c None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.165686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274781-22c2-441f-9d2f-74e0bf60823c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.165847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274781-22c2-441f-9d2f-74e0bf60823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.188871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4d274781-22c2-441f-9d2f-74e0bf60823c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:07.190123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4d274781-22c2-441f-9d2f-74e0bf60823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:07.280839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:07.286957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 156/320] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:07.330419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029db9c-e41d-4d60-a2cf-2163de28b1c1 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.330725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029db9c-e41d-4d60-a2cf-2163de28b1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.330959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029db9c-e41d-4d60-a2cf-2163de28b1c1 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.332716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 165/321] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:07.338071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.338405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.338693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.357950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:07.359552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:07.441182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 157/322] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:07.572850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029db9c-e41d-4d60-a2cf-2163de28b1c1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:07.703862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029db9c-e41d-4d60-a2cf-2163de28b1c1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:07.739285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:07.752859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:07.753658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.753930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.754200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.758175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:07.759687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:07.762181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 166/323] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:07.769181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:07.852891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:07.881225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583417b5-fa0e-48d6-b294-40f4fc8ddf33 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:07.911602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81320e-a15d-40a1-8882-6c021ab0b625 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:07.923658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:07.935246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34e27b-2ac9-47b0-9c2c-bed64154a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:07.936191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34e27b-2ac9-47b0-9c2c-bed64154a2ce None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.936626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34e27b-2ac9-47b0-9c2c-bed64154a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.937072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34e27b-2ac9-47b0-9c2c-bed64154a2ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.954403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34e27b-2ac9-47b0-9c2c-bed64154a2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-dd34e27b-2ac9-47b0-9c2c-bed64154a2ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:07.963963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:07.965120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:07.966055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 158/324] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:07.967812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107751c7-3ab4-49b1-87d1-76d37fd0f179 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.968355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107751c7-3ab4-49b1-87d1-76d37fd0f179 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.968763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486af27c-131f-4ea5-b645-69f05bfe6fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:07.968920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107751c7-3ab4-49b1-87d1-76d37fd0f179 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.969393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486af27c-131f-4ea5-b645-69f05bfe6fe0 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:07.969755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486af27c-131f-4ea5-b645-69f05bfe6fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:07.970049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486af27c-131f-4ea5-b645-69f05bfe6fe0 None admin] PATH_INFO: `/v3/groups` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:07.970688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486af27c-131f-4ea5-b645-69f05bfe6fe0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:07.972545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 159/325] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:07.974331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107751c7-3ab4-49b1-87d1-76d37fd0f179 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:07.976319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486af27c-131f-4ea5-b645-69f05bfe6fe0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:08.078602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107751c7-3ab4-49b1-87d1-76d37fd0f179 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:08.080009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 167/326] 10.222.0.65 () {60 vars in 1134 bytes} [Mon Aug 29 21:04:07 2022] POST /identity/v3/groups => generated 226 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:04:08.081588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:08.082032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:08.082311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:08.104658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:08.105962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:08.143641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:08.144332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:08.144719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:08.144988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:08.145554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:08.145858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:08.147421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:08.157676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 160/327] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:08.167247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:08.196310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 168/328] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:08.232344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:08.295582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a1706-f304-4f72-b969-88ea20deaa23 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:08.344820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:08.345827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:08.346300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:08.346625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] PATH_INFO: `/v3/users/a9936b18c5834a5a9088df1fc46dc79b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:08.348419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] RBAC: Authorizing `identity:get_user(user_id=a9936b18c5834a5a9088df1fc46dc79b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:08.350827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:08.353829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 161/329] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:08 2022] GET /identity/v3/users/a9936b18c5834a5a9088df1fc46dc79b => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:08.361141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:08.438213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:08.473245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:08.523609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e3093-4d35-4e67-b3c2-01ad322eeb0d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:08.577957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:08.578603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:08.578890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:08.579198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:08.580054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] RBAC: Adding query filter params (role.id=87804c58f0314370a20ec138295f6863, user.id=a9936b18c5834a5a9088df1fc46dc79b, scope.system=all) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:08.580362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:08.581986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:08.596235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 169/330] 10.222.0.65 () {56 vars in 1308 bytes} [Mon Aug 29 21:04:08 2022] GET /identity/v3/role_assignments?user.id=a9936b18c5834a5a9088df1fc46dc79b&scope.system=all&role.id=87804c58f0314370a20ec138295f6863 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:08.639128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfc14f-4689-4f05-9d68-c625ebb37ada None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:08.674515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:08.708701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:08.710443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:08.711045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:08.711534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:08.730035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:08.737846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:08.738832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:08.739655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:08.742889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 162/331] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:08.752955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:08.893384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c8a22-5249-4973-8965-9566fe569ae3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:08.907349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:08.936326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:08.937448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:08.938175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:08.938776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:08.940890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:08.941175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:08.942623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:08.957511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 170/332] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:08.966394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:09.135856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7406fc-d727-459f-8e25-ebe85d3ea886 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:09.197054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3f239-658e-4d05-98a3-b8528fd327a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:09.198350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3f239-658e-4d05-98a3-b8528fd327a1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:09.198982 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3f239-658e-4d05-98a3-b8528fd327a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:09.199463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3f239-658e-4d05-98a3-b8528fd327a1 None admin] PATH_INFO: `/v3/users/80784962154443a88dbde1e0e0560359` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:09.200886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3f239-658e-4d05-98a3-b8528fd327a1 None admin] RBAC: Authorizing `identity:get_user(user_id=80784962154443a88dbde1e0e0560359)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:09.202559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3f239-658e-4d05-98a3-b8528fd327a1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:09.204735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 163/333] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:08 2022] GET /identity/v3/users/80784962154443a88dbde1e0e0560359 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:09.211226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:09.339428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d45ed-c233-42e2-bff2-9149f27899d6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:09.387239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c820a8-fb30-4424-b4ac-a39937816060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:09.387944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c820a8-fb30-4424-b4ac-a39937816060 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:09.388284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c820a8-fb30-4424-b4ac-a39937816060 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:09.388575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c820a8-fb30-4424-b4ac-a39937816060 None admin] PATH_INFO: `/v3/system/users/80784962154443a88dbde1e0e0560359/roles/a84e6592db084ce2b9667f2cd70fdcb2` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:09.390055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c820a8-fb30-4424-b4ac-a39937816060 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=80784962154443a88dbde1e0e0560359, role_id=a84e6592db084ce2b9667f2cd70fdcb2)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:09.391460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c820a8-fb30-4424-b4ac-a39937816060 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:09.482344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 171/334] 10.222.0.65 () {58 vars in 1334 bytes} [Mon Aug 29 21:04:09 2022] PUT /identity/v3/system/users/80784962154443a88dbde1e0e0560359/roles/a84e6592db084ce2b9667f2cd70fdcb2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:10.756765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821140c-4590-4cf4-85d7-c7787200d275 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:10.757310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821140c-4590-4cf4-85d7-c7787200d275 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:10.757310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821140c-4590-4cf4-85d7-c7787200d275 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:10.758848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 164/335] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:10.765624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee4381-b741-4815-a2e8-b34153040748 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:10.766115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee4381-b741-4815-a2e8-b34153040748 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:10.766228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee4381-b741-4815-a2e8-b34153040748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:10.797228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-27ee4381-b741-4815-a2e8-b34153040748 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:10.799131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-27ee4381-b741-4815-a2e8-b34153040748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:10.841536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:10.898191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 172/336] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:10.977995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7173-2bc9-4b5c-91de-6edb9d3edd18 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:10.978341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7173-2bc9-4b5c-91de-6edb9d3edd18 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:10.979079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7173-2bc9-4b5c-91de-6edb9d3edd18 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:10.980737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 165/337] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:10.988126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:10.988451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:10.988680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:11.013245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:11.014302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:11.105284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 173/338] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:11.464102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7173-2bc9-4b5c-91de-6edb9d3edd18 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:11.530276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:11.598352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7173-2bc9-4b5c-91de-6edb9d3edd18 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:11.640942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:11.641584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:11.641863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:11.642136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:11.659164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:11.661497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:11.675381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:11.676357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:11.677168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:11.680267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 166/339] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:11.688739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:11.866798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:11.878099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898d71a-98e0-46e3-bd0a-00ee7ca72d55 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:11.928662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:11.929553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:11.929951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:11.930355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:11.955363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afb59e-7ccf-4e98-88e4-406c93d4a3bf None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:11.968326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:11.973272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:11.981558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:11.985382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:11.986187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 174/340] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:11.993691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:12.060042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:12.060659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:12.060950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:12.061231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:12.061973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:12.062317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:12.063811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:12.091912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 167/341] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:12.093758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:12.179054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5203e5-9c25-4203-8a48-6e5616a3234e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:12.231922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:12.232962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:12.233722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:12.234200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:12.235844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:12.240824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:12.242548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51c11c-9226-4bf7-81dd-aad097e6bb65 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:12.275918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beca9d-ccca-4d81-9de5-25357ad01d3a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:12.277721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 175/342] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:12.280542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edae42f-32ba-49ba-81d2-c35d662ba283 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:12.281410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edae42f-32ba-49ba-81d2-c35d662ba283 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:12.287913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edae42f-32ba-49ba-81d2-c35d662ba283 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:12.289916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 176/343] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:12.298428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edae42f-32ba-49ba-81d2-c35d662ba283 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:12.332106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0cf06d-d0e8-4d96-8ad6-b18bffef8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:12.332106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cf06d-d0e8-4d96-8ad6-b18bffef8337 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:12.332574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cf06d-d0e8-4d96-8ad6-b18bffef8337 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:12.332979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0cf06d-d0e8-4d96-8ad6-b18bffef8337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:12.374134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:12.482055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:12.506990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edae42f-32ba-49ba-81d2-c35d662ba283 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:12.580315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:12.581854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:12.583581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:12.584405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:12.591823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cf06d-d0e8-4d96-8ad6-b18bffef8337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:12.593899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0cf06d-d0e8-4d96-8ad6-b18bffef8337 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:12.863380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:12.880026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:12.944486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:12.993637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 168/344] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 898 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:12.995159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcea58-82a5-4cc1-ae66-27a35dc9906b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:12.995445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcea58-82a5-4cc1-ae66-27a35dc9906b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:12.995715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcea58-82a5-4cc1-ae66-27a35dc9906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.029787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-bfbcea58-82a5-4cc1-ae66-27a35dc9906b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:13.031942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bfbcea58-82a5-4cc1-ae66-27a35dc9906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:13.201570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:13.206785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 177/345] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:13.207902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:13.208670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 169/346] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:13.210059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc94149-5fd3-4b32-b916-929132c200ac None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.210340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc94149-5fd3-4b32-b916-929132c200ac None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.210616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc94149-5fd3-4b32-b916-929132c200ac None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.211910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 170/347] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:13.217059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc94149-5fd3-4b32-b916-929132c200ac None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:13.336999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078df609-313f-4a83-a10d-69ff8e6c8f0d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:13.358466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:13.361019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc94149-5fd3-4b32-b916-929132c200ac None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:13.378802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a151c-30bc-4b9b-808c-9cc49fa606c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:13.379595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a151c-30bc-4b9b-808c-9cc49fa606c7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.379776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a151c-30bc-4b9b-808c-9cc49fa606c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.380067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a151c-30bc-4b9b-808c-9cc49fa606c7 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.381320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a151c-30bc-4b9b-808c-9cc49fa606c7 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:13.382904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a151c-30bc-4b9b-808c-9cc49fa606c7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:13.387306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 178/348] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:13 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:13.387306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.387306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.392124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.413000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:13.414514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:13.419783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf6774-3a61-42b8-8734-74ed0fc7f1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:13.420825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf6774-3a61-42b8-8734-74ed0fc7f1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.421303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf6774-3a61-42b8-8734-74ed0fc7f1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.421766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf6774-3a61-42b8-8734-74ed0fc7f1c5 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.423160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf6774-3a61-42b8-8734-74ed0fc7f1c5 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:13.424953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf6774-3a61-42b8-8734-74ed0fc7f1c5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:13.427312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 171/349] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:13 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:13.432060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef5b63-6c2a-454b-b567-402e31bca34d None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.432432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef5b63-6c2a-454b-b567-402e31bca34d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.432783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef5b63-6c2a-454b-b567-402e31bca34d None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.434334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 172/350] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:13.435395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce3828-1931-4018-be11-4ea37f00c8cb None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.435821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce3828-1931-4018-be11-4ea37f00c8cb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.436211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce3828-1931-4018-be11-4ea37f00c8cb None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.437482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 173/351] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:13.438321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce3828-1931-4018-be11-4ea37f00c8cb None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:13.534377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:13.537464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 179/352] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:13.538916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:13.618152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce3828-1931-4018-be11-4ea37f00c8cb None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:13.673710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc4145-e424-4a88-8a81-cc8a2f128819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:13.675622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc4145-e424-4a88-8a81-cc8a2f128819 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.676785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc4145-e424-4a88-8a81-cc8a2f128819 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.677503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc4145-e424-4a88-8a81-cc8a2f128819 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.692706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc4145-e424-4a88-8a81-cc8a2f128819 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:13.695344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc4145-e424-4a88-8a81-cc8a2f128819 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:13.698912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 174/353] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:13 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:13.705286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.705980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.706686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.721427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f62003-72f7-4144-a195-a980a9c3caf8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:13.742844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:13.744267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:13.750948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:13.801800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0d0b2-45c7-488f-a8b8-453b103f68e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:13.804190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0d0b2-45c7-488f-a8b8-453b103f68e0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.804497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0d0b2-45c7-488f-a8b8-453b103f68e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.804781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0d0b2-45c7-488f-a8b8-453b103f68e0 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.807459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0d0b2-45c7-488f-a8b8-453b103f68e0 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:13.809655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0d0b2-45c7-488f-a8b8-453b103f68e0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:13.812858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 180/354] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:13 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:13.816868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:13.817325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:13.817772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:13.848556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 175/355] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:13.849731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:13.855365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:13.856838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:13.948783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:13.976027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 181/356] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:13.977684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.008088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e043aa1-edf7-4e52-91da-1814ce71d635 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.065417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.066274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.066760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.067287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.081059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:14.087407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:14.088369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:14.089298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 176/357] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:14.090039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.160266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4537946-1d17-4254-9d2d-94683c445dfb None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.179974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:14.217512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.218055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.218338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.218613 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.219351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=87804c58f0314370a20ec138295f6863, scope.project.id=263e684247604a1a8e174c46e3574118) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:14.219681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:14.222244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:14.229485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 182/358] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:13 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=87804c58f0314370a20ec138295f6863 => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:14.230196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.291106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f38a9-3e1c-44f0-b3f8-788f5d2217c3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.320197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:14.349948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.350601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.350886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.353107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.354210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=263e684247604a1a8e174c46e3574118) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:14.354525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:14.356137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:14.366660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 177/359] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:14.366660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.439015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a29ad-dc6b-4809-b62a-87d7c0607046 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.522205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77062e-072f-44f2-8915-e7be67f63abb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.524667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.526023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.526393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.526661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.544992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:14.553405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:14.554386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 21:04:14.555193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:14.558529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 183/360] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:14.559784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.571058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.571692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.575127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.575429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.576386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:14.576581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:14.577924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:14.590460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 178/361] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:14.591369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.621968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:14.653427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:14.708544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12866b5b-e4de-4a8f-b20a-a010a3395d6e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.742127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c51a6-7b3d-4df3-9a3b-90316dfa5350 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.767447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.768137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.768425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.768754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.789151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:14.790634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:14.791655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:14.792136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:14.792421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:14.795209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:14.796301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 184/362] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:14.797201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.805909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:14.813357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:14.814306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:14.815096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 179/363] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:14.815865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:14.960735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:14.968363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd2c1f-2455-46d2-9909-e49b89843651 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.989353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a3d40-6761-4cba-961a-2a45409a4a1b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:14.993161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:15.031212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.032313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.032753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.033190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.034246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:15.034722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.037024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.051292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 185/364] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:15.052615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.059774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.060464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.060744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.061021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.150941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.155920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.223415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be22c4f-cac9-455e-8a19-00f7e506c95f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:15.270107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.270902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.271244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.271695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.272476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:15.272851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.274328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.283632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 186/365] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:15.284518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.324083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:15.329370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 180/366] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:15.329370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.332217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:15.430647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450714cd-6438-4c30-be87-1fac055961f4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:15.476433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.477071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.477543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.477827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.479029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:15.479330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.481093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.487489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c8ba8-2715-4487-a0b3-b3fdb0c73a10 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:15.500653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 187/367] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:15.502069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.534712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.535741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.536251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.536721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] PATH_INFO: `/v3/users/80784962154443a88dbde1e0e0560359` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.538934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] RBAC: Authorizing `identity:get_user(user_id=80784962154443a88dbde1e0e0560359)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.541467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.545183 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 181/368] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/users/80784962154443a88dbde1e0e0560359 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:15.546319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.655798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:15.662081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:15.680038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09db9e-0c88-44a5-8e80-5ff6dd7459b7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:15.698265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5be7b-83ec-4c1f-8e78-669e7e22f8d0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:15.720213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.720860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.721144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.721417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.747920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.748599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.748908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.749662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.751082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.752753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.754912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 182/369] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:15.755709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.849922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:15.851500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:15.913658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ae33d-3da8-4bc7-b69f-5b0bacef3033 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:15.960936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:15.970832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:15.971903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:15.972370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:15.972871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:15.982615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 188/370] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:15.983838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:15.992469 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:16.115616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.118169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.146111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceb1e7-05b4-4bcb-ac9c-5335f8954990 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:16.195784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:16.196371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:16.197189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:16.197467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:16.198206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] RBAC: Adding query filter params (role.id=a84e6592db084ce2b9667f2cd70fdcb2, user.id=80784962154443a88dbde1e0e0560359, scope.system=all) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:16.199088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.201217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.218722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 189/371] 10.222.0.65 () {56 vars in 1308 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/role_assignments?user.id=80784962154443a88dbde1e0e0560359&scope.system=all&role.id=a84e6592db084ce2b9667f2cd70fdcb2 => generated 510 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:16.219780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.311181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 183/372] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:16.311802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.336451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:16.414783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:16.459313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc3c20-7173-4d5b-acb2-60315b4db504 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:16.512315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:16.515790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715a1d6-0c92-4bb4-8fb5-943cfec873d3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:16.519553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:16.520059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:16.520528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:16.524293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.526848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.531159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 190/373] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:16.532436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.606869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:16.608279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:16.608712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:16.609741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:16.613791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.617655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.622427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 184/374] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:16.628007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.696206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458e0cb-5dce-488f-805e-767a7f74d98a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:16.710247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:16.747929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:16.748830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:16.750141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:16.750141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] PATH_INFO: `/v3/projects/eb1de965aba54ca98b770d7165c4b67e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:16.752078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb1de965aba54ca98b770d7165c4b67e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.753975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.757073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 191/375] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/projects/eb1de965aba54ca98b770d7165c4b67e => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:16.758069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.764500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea33a8-77d2-4858-a5da-d37b6f42c33f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:16.778604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:16.812369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e112a-32bc-454b-b621-5821879b263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:16.813581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e112a-32bc-454b-b621-5821879b263d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:16.814030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e112a-32bc-454b-b621-5821879b263d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:16.814452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e112a-32bc-454b-b621-5821879b263d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:16.815372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e112a-32bc-454b-b621-5821879b263d None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=87804c58f0314370a20ec138295f6863, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:16.815836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e112a-32bc-454b-b621-5821879b263d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.817491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e112a-32bc-454b-b621-5821879b263d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.825367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 185/376] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=87804c58f0314370a20ec138295f6863 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:16.826606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e112a-32bc-454b-b621-5821879b263d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.895845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fc394-9703-46b5-8b94-34467dd1bcd9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:16.942789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:16.943584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:16.943855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:16.944320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:16.946604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:16.948476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:16.950300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 192/377] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:16.956629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:16.971234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e112a-32bc-454b-b621-5821879b263d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:17.017007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:17.020557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aa025-ceed-49c5-8156-7a0c19b6f4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:17.021925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aa025-ceed-49c5-8156-7a0c19b6f4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.022640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aa025-ceed-49c5-8156-7a0c19b6f4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.023361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aa025-ceed-49c5-8156-7a0c19b6f4a6 None admin] PATH_INFO: `/v3/groups/9e473a607f5447b6869351869450a753` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.031988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aa025-ceed-49c5-8156-7a0c19b6f4a6 None admin] RBAC: Authorizing `identity:get_group(group_id=9e473a607f5447b6869351869450a753)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:17.033804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aa025-ceed-49c5-8156-7a0c19b6f4a6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:17.036112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 186/378] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/groups/9e473a607f5447b6869351869450a753 => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:17.037672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e554c-d1c9-4578-90d0-810501a7320f None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.038160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e554c-d1c9-4578-90d0-810501a7320f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.038676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e554c-d1c9-4578-90d0-810501a7320f None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.042219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 187/379] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:17.048954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.049521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.049999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.074901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:17.078028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:17.127485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:17.147172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ca55b-5a52-48b9-9aea-bee287b6ee89 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:17.198185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 188/380] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:17.200929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:17.224191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:17.225198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.225652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.226102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.227361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:17.228503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:17.231112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7b838-e8f7-4e4e-a560-f48422ffa081 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:17.243537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 193/381] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:16 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 260 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:17.245641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62560d-704f-4802-bcfc-82f1130ee286 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.246094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62560d-704f-4802-bcfc-82f1130ee286 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.246539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62560d-704f-4802-bcfc-82f1130ee286 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.248933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 194/382] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:17.258700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.259456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.260142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.292648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:17.295001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:17.405799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6b3da-7b14-4cd2-8811-a175001f8616 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:17.428020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 195/383] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:17.468463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:17.469115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.469396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.469674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.470700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] RBAC: Adding query filter params (group.id=9e473a607f5447b6869351869450a753, role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=eb1de965aba54ca98b770d7165c4b67e) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:17.471016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:17.472600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:17.480190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 189/384] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:17 2022] GET /identity/v3/role_assignments?group.id=9e473a607f5447b6869351869450a753&scope.project.id=eb1de965aba54ca98b770d7165c4b67e&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 260 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:17.632840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:17.713326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:17.808224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18aab7-c566-4b2b-9748-aead3c256a1b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:17.867625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:17.872035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:17.873057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:17.873482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:17.873910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:17.887477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:17.892813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:17.893967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:17.895030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:17.897097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 196/385] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:17.903827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:17.933782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:18.071128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fbc83-ee77-4ce9-b663-5742680d1db7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:18.082838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:18.086162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239665a7-3449-4108-b385-3c1ecb4d8158 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:18.132253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:18.133844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:18.134454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:18.135624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:18.150179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:18.158106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:18.159082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:18.159867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:18.162803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 190/386] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:18.173249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:18.175335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:18.176997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:18.177464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:18.177921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:18.179041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:18.179610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:18.182644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:18.211223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 197/387] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:18.217428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:18.411031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:18.428695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6256fe0-a5de-4163-9a32-9bfcb7a3a210 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:18.478507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da978b01-9ad1-417d-9e57-244fb32368dd None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:18.541337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:18.542078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:18.542486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:18.542775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:18.543839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:18.544372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:18.545869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:18.548044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:18.549344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:18.550067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:18.550801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:18.552849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:18.555750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:18.558686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 198/388] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:18 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:18.568937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:18.574992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 191/389] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:18.580588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:18.620506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:18.739041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45d6a3-fd84-43e7-903f-0ca6f03865c9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:18.796746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb81fc-9b00-42b4-a656-5b5493cd0a98 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:18.829569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:18.830236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:18.830524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:18.830808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:18.833491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:18.835132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:18.837844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 199/390] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:18 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:18.843985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:18.845605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:18.846755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:18.847249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:18.847722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:18.849530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:18.851637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:18.854762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 192/391] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:18 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:18.860433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:18.896332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:19.060018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303c184-b91c-438b-8251-68bb48b2c8f5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:19.072392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:19.076836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4124a6-f86b-4ba7-b3fb-d827548fb3ea None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:19.129720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcaa82-fb99-4b80-a260-65a49879ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:19.130171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcaa82-fb99-4b80-a260-65a49879ed34 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:19.130454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcaa82-fb99-4b80-a260-65a49879ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:19.130735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcaa82-fb99-4b80-a260-65a49879ed34 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:19.132800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcaa82-fb99-4b80-a260-65a49879ed34 None admin] RBAC: Authorizing `identity:create_grant(project_id=263e684247604a1a8e174c46e3574118, group_id=170fb803f44a49f8a52d54c4fd95f5b9, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:19.134822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcaa82-fb99-4b80-a260-65a49879ed34 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:19.157327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:19.157984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:19.158296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:19.158683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:19.160312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:19.161866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:19.165375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 193/392] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:18 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:19.171233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:19.211765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 200/393] 10.222.0.65 () {58 vars in 1442 bytes} [Mon Aug 29 21:04:18 2022] PUT /identity/v3/projects/263e684247604a1a8e174c46e3574118/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:19.276677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:19.290426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390be681-3b59-45a5-85af-142b30bdfcc8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:19.342082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61686941-4688-4d0e-8503-ae352bdecbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:19.342954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686941-4688-4d0e-8503-ae352bdecbdd None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:19.343276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686941-4688-4d0e-8503-ae352bdecbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:19.343640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686941-4688-4d0e-8503-ae352bdecbdd None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:19.349499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61686941-4688-4d0e-8503-ae352bdecbdd None admin] RBAC: Authorizing `identity:create_grant(project_id=263e684247604a1a8e174c46e3574118, group_id=170fb803f44a49f8a52d54c4fd95f5b9, role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:19.351195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61686941-4688-4d0e-8503-ae352bdecbdd None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:19.434109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 194/394] 10.222.0.65 () {58 vars in 1442 bytes} [Mon Aug 29 21:04:19 2022] PUT /identity/v3/projects/263e684247604a1a8e174c46e3574118/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/c2a3834a436b40deabcb87f412ca5322 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:19.649827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e0409-0b70-4ccb-901c-f18be32b5563 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:19.650228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e0409-0b70-4ccb-901c-f18be32b5563 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:19.650571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e0409-0b70-4ccb-901c-f18be32b5563 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:19.652176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 201/395] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:19.660194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546a969-2064-4768-9004-10a2c5c055df None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:19.661192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546a969-2064-4768-9004-10a2c5c055df None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:19.661684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546a969-2064-4768-9004-10a2c5c055df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:19.686202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d546a969-2064-4768-9004-10a2c5c055df None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:19.687981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d546a969-2064-4768-9004-10a2c5c055df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:19.790944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 195/396] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:20.020884 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a474f5-94b2-46fc-a4d5-ad71a4b22d70 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.021178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a474f5-94b2-46fc-a4d5-ad71a4b22d70 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.021395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a474f5-94b2-46fc-a4d5-ad71a4b22d70 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.022961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 202/397] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:20.032023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd548ddb-991e-4c60-b8c2-761e38de2185 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.032477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd548ddb-991e-4c60-b8c2-761e38de2185 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.032897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd548ddb-991e-4c60-b8c2-761e38de2185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.061336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cd548ddb-991e-4c60-b8c2-761e38de2185 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:20.062784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cd548ddb-991e-4c60-b8c2-761e38de2185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:20.160047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 196/398] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:20.202224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a474f5-94b2-46fc-a4d5-ad71a4b22d70 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:20.353354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0b9c-fb1c-4569-bf1e-3c03cf6ef6bd None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.353812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0b9c-fb1c-4569-bf1e-3c03cf6ef6bd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.354026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece0b9c-fb1c-4569-bf1e-3c03cf6ef6bd None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.360145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 197/399] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:20.364020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.364303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.364577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.377958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a474f5-94b2-46fc-a4d5-ad71a4b22d70 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:20.398951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:20.400321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:20.403689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:20.435105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:20.436007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.436330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.436619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.454855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:20.463318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:20.466542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:20.467633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 203/400] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:20.472566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:20.516467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:20.569868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 198/401] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:20.570980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:20.717882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6cfbe-b933-41f4-95e3-3fe93b86197c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:20.800135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:20.801164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.801649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.802137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.802851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:20.803157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:20.805037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:20.829198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 204/402] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:20.834533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:20.865445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8320a-c897-4e88-a776-d2de848a924d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:20.965672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:20.966664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:20.967168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:20.967659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:20.981537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:20.987064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:20.993216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:20.995196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 199/403] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:04:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:20.997042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.039812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:21.074046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84effb69-328b-43c9-a07f-b9a9578a330a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.139423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.140159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.140444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.140815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.142009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.143539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.150086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 205/404] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:20 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:21.151055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.166872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b7795-5873-4fd3-b82b-b55398d3e8c3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.225674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.228137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.228733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.229285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.250316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:21.255453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:21.256586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 200/405] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:21.257409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.299657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1093e-9278-4f3f-9144-243523840cc3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.357378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.359752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.360570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.361419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.362304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:21.362909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.366059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.379622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:21.381637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 206/406] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:21.383286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.399470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:21.450311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad21cda-1493-48a5-bed9-276a715a4329 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.512888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.514608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.515653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.516274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.518484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.521932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.525078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 201/407] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:21 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:21.526141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.552904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eced94-89b0-4f2c-9f19-e9749e225479 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.598890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.600113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.600593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.601129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.602350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:21.602520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.604072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.614288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 207/408] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:21.615618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.685586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577821-99e3-4437-8082-76bfffe3c812 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.726075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:21.731455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:21.738857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.740410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.741148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.741868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] PATH_INFO: `/v3/groups/9e473a607f5447b6869351869450a753` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.744639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] RBAC: Authorizing `identity:get_group(group_id=9e473a607f5447b6869351869450a753)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.747711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.750404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 202/409] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:21 2022] GET /identity/v3/groups/9e473a607f5447b6869351869450a753 => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:21.752832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:21.781250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13912fcf-baa9-42a3-9c78-a47b9e8a9ee8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.837698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a94f9a-f8de-4fac-baa7-ff86535d2b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.838394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a94f9a-f8de-4fac-baa7-ff86535d2b22 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.838703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a94f9a-f8de-4fac-baa7-ff86535d2b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.839084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a94f9a-f8de-4fac-baa7-ff86535d2b22 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.841961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a94f9a-f8de-4fac-baa7-ff86535d2b22 None admin] RBAC: Authorizing `identity:create_grant(project_id=23412b7c545d48c5b5fce56d6eefb33e, group_id=170fb803f44a49f8a52d54c4fd95f5b9, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.844076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a94f9a-f8de-4fac-baa7-ff86535d2b22 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.906405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2ff38-79ad-40cb-a7aa-2a16fccf0e17 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:21.952972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:21.953612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:21.953888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:21.954160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:21.955525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:21.957039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:21.959082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 203/410] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:21 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:21.960068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:22.019502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 208/411] 10.222.0.65 () {58 vars in 1442 bytes} [Mon Aug 29 21:04:21 2022] PUT /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:22.020890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e6526-1b36-4b63-86eb-0966a5fb165c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.021170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e6526-1b36-4b63-86eb-0966a5fb165c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.021512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e6526-1b36-4b63-86eb-0966a5fb165c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.022865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 209/412] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:22.024075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c9939-62eb-45bc-b7f8-2e44c00a8f2e None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.024364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c9939-62eb-45bc-b7f8-2e44c00a8f2e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.024960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c9939-62eb-45bc-b7f8-2e44c00a8f2e None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.026874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 210/413] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:22.030847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c9939-62eb-45bc-b7f8-2e44c00a8f2e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:22.062528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:22.129778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b911c-dc8d-43db-a58b-9b48a07b9673 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:22.195109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85a01b-7724-43a8-8fbb-ae6ea54840df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:22.195768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85a01b-7724-43a8-8fbb-ae6ea54840df None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.196095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85a01b-7724-43a8-8fbb-ae6ea54840df None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.196435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85a01b-7724-43a8-8fbb-ae6ea54840df None admin] PATH_INFO: `/v3/projects/eb1de965aba54ca98b770d7165c4b67e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.198158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85a01b-7724-43a8-8fbb-ae6ea54840df None admin] RBAC: Authorizing `identity:get_project(project_id=eb1de965aba54ca98b770d7165c4b67e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:22.199551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85a01b-7724-43a8-8fbb-ae6ea54840df None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:22.202639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 204/414] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:21 2022] GET /identity/v3/projects/eb1de965aba54ca98b770d7165c4b67e => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:22.203029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.203315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.203587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.229802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:22.230150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c9939-62eb-45bc-b7f8-2e44c00a8f2e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:22.231681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:22.281161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:22.290586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cea2f-1bb2-4875-b640-52fa2e0ef5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:22.291646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cea2f-1bb2-4875-b640-52fa2e0ef5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.292156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cea2f-1bb2-4875-b640-52fa2e0ef5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.292639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cea2f-1bb2-4875-b640-52fa2e0ef5d8 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.308260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cea2f-1bb2-4875-b640-52fa2e0ef5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:22.308260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cea2f-1bb2-4875-b640-52fa2e0ef5d8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:22.308260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 211/415] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:22 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:22.308260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.308260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.308260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.342992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:22.344551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:22.346833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 205/416] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:22.353925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:22.482968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 212/417] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:22.485425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:22.485961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:22.566063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c76fc15-931d-41fa-9341-8322d77d6b80 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:22.670254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:22.671237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.671725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.672216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] PATH_INFO: `/v3/projects/eb1de965aba54ca98b770d7165c4b67e/groups/9e473a607f5447b6869351869450a753/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.676149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb1de965aba54ca98b770d7165c4b67e, group_id=9e473a607f5447b6869351869450a753, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:22.678705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:22.777126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc552d-6966-47ab-93d8-d8978a2be844 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:22.782272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 206/418] 10.222.0.65 () {58 vars in 1442 bytes} [Mon Aug 29 21:04:22 2022] PUT /identity/v3/projects/eb1de965aba54ca98b770d7165c4b67e/groups/9e473a607f5447b6869351869450a753/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:22.838562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:22.850305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:22.851710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:22.853017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:22.853569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:22.853996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:22.858603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=23412b7c545d48c5b5fce56d6eefb33e, group_id=170fb803f44a49f8a52d54c4fd95f5b9, role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:22.863577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:22.977581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 213/419] 10.222.0.65 () {58 vars in 1442 bytes} [Mon Aug 29 21:04:22 2022] PUT /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e/groups/170fb803f44a49f8a52d54c4fd95f5b9/roles/c2a3834a436b40deabcb87f412ca5322 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:23.002193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe7e80-5235-4156-93b3-59fe21a8b295 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:23.031382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:23.050238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:23.051793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:23.052310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:23.052811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:23.068666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:23.077880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:23.078875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:23.079679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:23.082681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:23.082861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 207/420] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:23.090235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:23.263254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd40ca-8ffe-43ce-bb5d-17eff283e1c5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:23.273467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26a9f6-f2e2-43b2-b346-f628b677ca82 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:23.338479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aef15-8126-4014-890f-88cc740406db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:23.338479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aef15-8126-4014-890f-88cc740406db None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:23.338479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aef15-8126-4014-890f-88cc740406db None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:23.338944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aef15-8126-4014-890f-88cc740406db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:23.354500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aef15-8126-4014-890f-88cc740406db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c65aef15-8126-4014-890f-88cc740406db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:23.360579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:23.362422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:23.364228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 214/421] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:23.366319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:23.367590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:23.368332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:23.368999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:23.369794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aef15-8126-4014-890f-88cc740406db None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:23.370612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] RBAC: Adding query filter params (name=member) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:23.371334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:23.374084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:23.387491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 208/422] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:23.394616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:23.479923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:23.536602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:23.556922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4eae0d-9803-4a5d-b0c4-06ac0e9ed5af None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:23.642038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:23.642674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:23.642962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:23.643236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:23.650119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aef15-8126-4014-890f-88cc740406db None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:23.709258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:23.710117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:23.710412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:23.710853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:23.711627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:23.711986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:23.713745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:23.731544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 215/423] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:23.743559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:23.779250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:23.788000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:23.926943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd71cb-c02e-43f6-b685-c7f211c65362 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:23.947532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:23.960431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:23.974859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:23.975493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:23.976886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:23.977212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.100329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 209/424] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 21:04:24.110037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:24.182719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:24.185535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:24.252760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222d640-4d65-4391-b668-4baa0950c019 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:24.290477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:24.292082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.292594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.293064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.294602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:24.296648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:24.298548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 210/425] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:24 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:24.308976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:24.331221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:24.382103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 216/426] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:24.390016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:24.437230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:24.443623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebcb25-c0bb-4c26-8b6e-e2685e4e3353 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:24.494821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:24.495867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.496377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.496934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.500652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:24.502427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:24.508127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 211/427] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:24 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:24.512943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:24.569685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3696532-5521-494c-8531-ee11a1b8d734 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:24.630153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:24.631444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.631559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.631861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] PATH_INFO: `/v3/projects/263e684247604a1a8e174c46e3574118` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.636817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] RBAC: Authorizing `identity:get_project(project_id=263e684247604a1a8e174c46e3574118)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:24.638289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:24.641096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 217/428] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:24 2022] GET /identity/v3/projects/263e684247604a1a8e174c46e3574118 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:24.646998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:24.647743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:24.677322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d6efd-bce1-4e54-9178-d6281fe264aa None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:24.725056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:24.726165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.726657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.727208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.728270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=87804c58f0314370a20ec138295f6863, scope.project.id=263e684247604a1a8e174c46e3574118) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:24.728854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:24.730614 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c273f-5cf1-4d97-8741-b7aaacb91012 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:24.738545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 212/429] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:24 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=87804c58f0314370a20ec138295f6863 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:24.810601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:24.823873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878b5e3-5214-44d3-a053-3b8bf00f27d2 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.824526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878b5e3-5214-44d3-a053-3b8bf00f27d2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.825029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878b5e3-5214-44d3-a053-3b8bf00f27d2 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.827096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 213/430] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:24.835154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.835459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.835711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.837421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f7ab-970e-438e-88a3-483ba9298e65 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:24.880995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:24.882297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:24.888894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:24.891097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:24.891714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:24.892266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:24.893860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:24.896420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:24.898856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 218/431] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:24 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:24.907184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:25.041489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 214/432] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:25.057225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31110f52-c140-4347-9989-bd74dc6b3764 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:25.100892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:25.102603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.103136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.103661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.104679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=263e684247604a1a8e174c46e3574118) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:25.105286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:25.107197 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:25.121089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 219/433] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:24 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=263e684247604a1a8e174c46e3574118&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:25.411300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:25.510974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:25.560303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5ea9a-3943-49ca-a96e-882c8f1ddd90 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:25.617311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770050a6-ec1b-4c09-9431-3f0892682933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:25.618008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770050a6-ec1b-4c09-9431-3f0892682933 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.618291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770050a6-ec1b-4c09-9431-3f0892682933 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.618673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770050a6-ec1b-4c09-9431-3f0892682933 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.632182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770050a6-ec1b-4c09-9431-3f0892682933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-770050a6-ec1b-4c09-9431-3f0892682933 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:25.639245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:25.640521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:25.642197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 215/434] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:25.649800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:25.674808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:25.770331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326fa2-79de-49b8-bae1-8e477e5abc73 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.770804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326fa2-79de-49b8-bae1-8e477e5abc73 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.771638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326fa2-79de-49b8-bae1-8e477e5abc73 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.773054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 216/435] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:25.775018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9efe3-6095-494b-bff6-b79e0f7c828f None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.775018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9efe3-6095-494b-bff6-b79e0f7c828f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.775018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9efe3-6095-494b-bff6-b79e0f7c828f None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.779036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 217/436] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:25.780127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.780413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.781340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.803811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:25.805164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:25.814466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8029200-1993-49ee-8b20-fe4f283f683a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:25.866147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:25.866997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.867411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.867714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.868490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] RBAC: Adding query filter params (name=member) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:25.868802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:25.870230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff17e0a-854e-4b72-a00b-9bc68060d174 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:25.882014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 220/437] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:04:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:25.884008 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:25.884306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:25.884583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:25.905344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:25.906593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:25.911237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 218/438] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:25.912212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.004324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.007042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 221/439] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:26.084688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ceef4-cbfa-4ca4-9446-f756adfeeb8c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.141570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.142293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.142574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.142917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.190445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.195095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.249293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:26.251862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:26.339231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4092c24-b246-4e2a-b2aa-4c6f19d4c3c1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.348762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.382421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.383609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.383906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.384175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.397892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:26.400069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 219/440] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:26.401199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:26.403974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:26.405118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:26.407513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 222/441] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:26.408885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.509266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.545314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a63385-22c4-4d70-aa25-bcff74f91a1d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.559354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e83740-00a9-4518-8bb7-da9415d52a2a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.586944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.587833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.588153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.588428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.604331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:26.609565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:26.611416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:26.613340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:26.615508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a0204-524c-4012-9c27-6151a013404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.615763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 220/442] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:26.615763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.616184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a0204-524c-4012-9c27-6151a013404c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.616184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a0204-524c-4012-9c27-6151a013404c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.616184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a0204-524c-4012-9c27-6151a013404c None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.616184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a0204-524c-4012-9c27-6151a013404c None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:26.616646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a0204-524c-4012-9c27-6151a013404c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:26.618785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 223/443] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:26.619692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a0204-524c-4012-9c27-6151a013404c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.643600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.741645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40ef3-cdbf-4b50-9851-9fc19e4087c4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.765100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a0204-524c-4012-9c27-6151a013404c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.782722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.783344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.783624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.783925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.784599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:26.784902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:26.786315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:26.798632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 221/444] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:26.799497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.802012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.809346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.809983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.818530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.818530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.818530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:26.818530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:26.818530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:26.822570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 224/445] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:26.823430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:26.918234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:26.938044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775b34d-bb0b-4025-ba2a-d5f446f64061 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.967916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769460cd-edd4-4205-ab6b-41ca7d7d233b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:26.996351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:26.997129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:26.997338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:26.997613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:26.998870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.000436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.002162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 222/446] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:27.003020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.032756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.033603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.034034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.034414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.115120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:27.125835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.128363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.155042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975e550-26b5-45be-a5ea-cca3f6c023ec None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:27.200600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.200911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.201186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.201554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.228919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:27.255388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 225/447] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:27.256336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.289707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.292047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.396965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:27.407288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc5336-05da-451f-aed2-e9dc1c026648 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:27.446098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 223/448] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:27.447128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.457846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.458435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.458723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.458997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.460580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=87804c58f0314370a20ec138295f6863, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:27.460580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.462461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.469411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 226/449] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=87804c58f0314370a20ec138295f6863 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:27.470754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.521475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:27.608985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02585e-ad1c-4069-b174-5cf785d5475d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:27.637370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ffd49-9b76-44d7-a130-fda26db3adfc None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:27.659627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.660369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.660653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.660932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] PATH_INFO: `/v3/projects/23412b7c545d48c5b5fce56d6eefb33e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.662138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=23412b7c545d48c5b5fce56d6eefb33e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.664878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.665758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 224/450] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/projects/23412b7c545d48c5b5fce56d6eefb33e => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:27.673813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.678945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.680125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.680551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.680835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] PATH_INFO: `/v3/projects/eb1de965aba54ca98b770d7165c4b67e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.682400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] RBAC: Authorizing `identity:get_project(project_id=eb1de965aba54ca98b770d7165c4b67e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.684027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.686711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 227/451] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/projects/eb1de965aba54ca98b770d7165c4b67e => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:27.693283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.726682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:27.806462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b71ec8-c074-4718-b1fe-c295dce26ea3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:27.828331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10240e0d-aec7-4588-8bf4-e5e8c60b0369 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:27.839041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:27.856586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.857585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.861052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.861052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] PATH_INFO: `/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.861052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] RBAC: Authorizing `identity:get_group(group_id=170fb803f44a49f8a52d54c4fd95f5b9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.861052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.863007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 225/452] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/groups/170fb803f44a49f8a52d54c4fd95f5b9 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:27.869772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.886208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:27.887559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:27.887987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:27.888137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] PATH_INFO: `/v3/groups/9e473a607f5447b6869351869450a753` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:27.889397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] RBAC: Authorizing `identity:get_group(group_id=9e473a607f5447b6869351869450a753)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:27.890904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:27.896026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 228/453] 10.222.0.65 () {56 vars in 1180 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/groups/9e473a607f5447b6869351869450a753 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:27.899358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:27.977951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d1d9-50e5-440c-8b90-92ac1e9f6c13 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:28.001689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:28.009039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d28db-c8ca-4704-80d4-319aa5ab51a2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:28.017418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:28.018199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:28.018481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:28.018782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:28.019577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] RBAC: Adding query filter params (group.id=170fb803f44a49f8a52d54c4fd95f5b9, role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=23412b7c545d48c5b5fce56d6eefb33e) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:28.019955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:28.021494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93d36b-fd7f-4088-9637-26ccd5456a06 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:28.027783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 226/454] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/role_assignments?group.id=170fb803f44a49f8a52d54c4fd95f5b9&scope.project.id=23412b7c545d48c5b5fce56d6eefb33e&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:28.048433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:28.049068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:28.049350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:28.049633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:28.050458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] RBAC: Adding query filter params (group.id=9e473a607f5447b6869351869450a753, role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=eb1de965aba54ca98b770d7165c4b67e) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:28.050770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:28.052343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20049b60-11c5-4b22-9765-54aceca74035 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:28.060166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 229/455] 10.222.0.65 () {56 vars in 1376 bytes} [Mon Aug 29 21:04:27 2022] GET /identity/v3/role_assignments?group.id=9e473a607f5447b6869351869450a753&scope.project.id=eb1de965aba54ca98b770d7165c4b67e&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:30.518847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dff210-2a1a-412e-9a48-14be92c38c1c None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:30.519530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dff210-2a1a-412e-9a48-14be92c38c1c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:30.519530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dff210-2a1a-412e-9a48-14be92c38c1c None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:30.521457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 227/456] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:30.527922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7e5d4-ab98-4b44-8d38-fd42a735aae9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:30.528622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7e5d4-ab98-4b44-8d38-fd42a735aae9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:30.528622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7e5d4-ab98-4b44-8d38-fd42a735aae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:30.561120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-65b7e5d4-ab98-4b44-8d38-fd42a735aae9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:30.562544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-65b7e5d4-ab98-4b44-8d38-fd42a735aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:30.678649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 230/457] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:31.254488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dff210-2a1a-412e-9a48-14be92c38c1c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:31.264174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917774-f3d6-4aac-93d4-d753ff92148f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.264464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917774-f3d6-4aac-93d4-d753ff92148f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.264744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917774-f3d6-4aac-93d4-d753ff92148f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.266518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 231/458] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:31.272648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de1d79-7e6c-4c2a-a550-0ce47c7b828a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.274001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de1d79-7e6c-4c2a-a550-0ce47c7b828a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.274503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de1d79-7e6c-4c2a-a550-0ce47c7b828a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.296122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-30de1d79-7e6c-4c2a-a550-0ce47c7b828a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:31.297592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-30de1d79-7e6c-4c2a-a550-0ce47c7b828a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:31.331417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:31.392341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dff210-2a1a-412e-9a48-14be92c38c1c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:31.402300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 232/459] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:31.404108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796b637-0c67-4e97-9542-f214dd233b26 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.404601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796b637-0c67-4e97-9542-f214dd233b26 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.405070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796b637-0c67-4e97-9542-f214dd233b26 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.406563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 233/460] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:31.412740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.413301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.414404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.436956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:31.438798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:31.455501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:31.486086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:31.486735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.487022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.487309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.514930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:31.516815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:31.519634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 234/461] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-2d61552e-f786-4c21-b867-9397cdebbbca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:31.543586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:31.544979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:31.545875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:31.545875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:31.545875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:31.553280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 228/462] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:31.559424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:31.695774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f705a-755d-49ca-b5c9-66e27c569a94 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.696091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f705a-755d-49ca-b5c9-66e27c569a94 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.696439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f705a-755d-49ca-b5c9-66e27c569a94 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.703222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 229/463] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:31.704557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.704841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.705119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.724610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3fd6c-076d-444a-9d7e-f0a183e7161d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:31.736765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:31.737968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:31.803433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:31.804173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:31.804456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:31.804726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:31.805422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:31.805726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:31.807105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:31.818523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 230/464] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:31.835075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 235/465] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:31.843941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:31.862412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:31.888260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:31.906058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:31.979087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5eb94-fe15-4053-9964-3b4d2f76c9b8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.003642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaefb27-5f46-49cc-a3fb-c6cd4edc956e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.009610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.010436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.010664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.010942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.011601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.013194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.046581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.047574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.048118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.048609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.063752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.065600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:32.081782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:32.083473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:32.084393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:32.084393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:32.084393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:32.089553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 236/466] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:32.089553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:32.155875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:32.204085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 231/467] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:04:31 2022] POST /identity/v3/users => generated 246 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:04:32.204085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:32.210762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2fa7e-da14-4660-b81e-3287bc089d54 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.268952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.269587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.270020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.270333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.288568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.290623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:32.324862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:32.325840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:32.326754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:32.326754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:32.326754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:32.332488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 237/468] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:32.334049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:32.383812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:32.397660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc06a78-aadd-44ef-85f3-a77d5d319e90 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.467952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.468645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.468930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.469226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.470025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:32.470332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.471980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.494096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725ae36-8ac3-48ab-ae9f-9395a2f9c5f9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.501286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 232/469] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:32.506015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:32.513927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:32.537359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.537918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.538200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.538508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.550270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.552421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-cd35c2df-693e-4cb7-9d65-7489c21b5eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:32.570834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:32.572576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:32.573777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:32.573777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:32.573777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:32.573777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 238/470] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:32.574428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddce48-4612-41fd-94e7-c48264439488 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.574696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddce48-4612-41fd-94e7-c48264439488 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.574975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daddce48-4612-41fd-94e7-c48264439488 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.581761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 239/471] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:32.581761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddce48-4612-41fd-94e7-c48264439488 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:32.643584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1a886-17b9-4903-b3f5-ac0db2518fe7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.685153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.685762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.686064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.686341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.686978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:32.687394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.688702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.708505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 233/472] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:32.708505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:32.716589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daddce48-4612-41fd-94e7-c48264439488 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.724545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:32.758991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1560f-727f-4d65-a871-fd46b225d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.759625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1560f-727f-4d65-a871-fd46b225d9ac None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.759930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1560f-727f-4d65-a871-fd46b225d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.760214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1560f-727f-4d65-a871-fd46b225d9ac None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.760873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1560f-727f-4d65-a871-fd46b225d9ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.762866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1560f-727f-4d65-a871-fd46b225d9ac None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.836009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c366e6-94be-4706-a7f2-5744a0716e54 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:32.852060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 240/473] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:04:32 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:04:32.853454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.853766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.854046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.889183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:32.890763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:32.902449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:32.903588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:32.904218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:32.904828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:32.905832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:32.906354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:32.909259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:32.910791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:32.939923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 234/474] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:32.939923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:33.022511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 241/475] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:33.024597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:33.049568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:33.108881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc78093-4287-47af-925d-b50ec6fcec45 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:33.161263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:33.162505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:33.163018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:33.163926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:33.164605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:33.166833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:33.214190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07d6ac-7173-4000-8113-1a6bb029c730 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:33.259833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:33.260542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:33.260822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:33.261095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:33.261829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:33.263638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:33.297247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 235/476] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:04:32 2022] POST /identity/v3/users => generated 248 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:04:33.353734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 242/477] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:04:33 2022] POST /identity/v3/users => generated 249 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:04:33.443933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:33.605956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc8734-460c-4a81-b6af-41c633e5cede None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:33.647301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:33.660762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:33.661827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:33.665788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:33.665788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:33.665788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:33.665788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:33.739739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.sql.core [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 21:04:33.739739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 21:04:33.739739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [parameters: {'id': '87957fff84734ef78429fe811f1663bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 21:04:33.739739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81875) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result.read() Aug 29 21:04:33.761877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:04:33.762865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application yield resource Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise self.value Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application session.commit() Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 21:04:33.763810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.session.flush() Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application compat.raise_( Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 21:04:33.765360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:04:33.767347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result.read() Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:04:33.769091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [parameters: {'id': '87957fff84734ef78429fe811f1663bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:33.770489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 21:04:33.772098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:33.774618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 236/478] 10.222.0.65 () {60 vars in 1131 bytes} [Mon Aug 29 21:04:33 2022] POST /identity/v3/roles => generated 144 bytes in 321 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:33.774758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:33.787200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:33.956231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79335c-d986-4231-9735-b35a6beff12a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:34.016006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ef251-7012-407a-9cdf-6caf3f6c8c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:34.017050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef251-7012-407a-9cdf-6caf3f6c8c7a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:34.017477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef251-7012-407a-9cdf-6caf3f6c8c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:34.017895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ef251-7012-407a-9cdf-6caf3f6c8c7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:34.041199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ef251-7012-407a-9cdf-6caf3f6c8c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4e2ef251-7012-407a-9cdf-6caf3f6c8c7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:34.055145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:34.056236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 21:04:34.057059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:34.066976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 243/479] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:04:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:34.067189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:34.238126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043ae38-b218-4da0-a3b6-796e83a333a9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:34.294111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:34.294646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:34.294928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:34.295202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:34.295916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:34.296223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:34.297632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559a304-7b12-43c5-85ba-d12a6d4a9109 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:34.308619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 237/480] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:34.633755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bc63f-3c4b-48c0-bafc-f7819885909a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:34.634401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bc63f-3c4b-48c0-bafc-f7819885909a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:34.634899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bc63f-3c4b-48c0-bafc-f7819885909a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:34.637316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 244/481] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:34.642971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:34.643521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:34.643879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:34.674379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:34.676313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:34.739111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:34.770979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 238/482] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:35.093079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bc63f-3c4b-48c0-bafc-f7819885909a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:35.195559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:35.254827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bc63f-3c4b-48c0-bafc-f7819885909a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:35.294505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:35.295495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:35.295953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:35.296240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:35.312267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:35.317659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:35.318577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:35.319834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 245/483] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:35.325690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:35.468895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8a2d0-83ed-4ba3-ab96-d7b32f8e2161 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:35.509218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:35.509850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:35.510128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:35.510438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:35.511106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:35.511409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:35.512855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3212151-7f97-4d74-a340-ec7bc09cdbe3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:35.522097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 239/484] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:35.529122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:35.708267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d6c9-8a70-4acf-8732-e0398c5e0ac6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:35.708487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c362f-b8b0-4666-bfda-f99e4f62ab43 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:35.708901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c362f-b8b0-4666-bfda-f99e4f62ab43 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:35.708901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c362f-b8b0-4666-bfda-f99e4f62ab43 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:35.710318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 240/485] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:04:35.718395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:35.718707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:35.719005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:35.757539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:35.759390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:35.760492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:35.775758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:35.777058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:35.779410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:35.779410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:35.798252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:35.800303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:35.833360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:35.834412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:35.835836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:35.835836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:35.835836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:35.838606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 246/486] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:35.844824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:35.883563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:35.953442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 241/487] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:35.979786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35b23c-bc10-42ee-a8d8-674e93b7ffc1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:36.017375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152b64a-5323-428e-a139-451125f67958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:36.017998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152b64a-5323-428e-a139-451125f67958 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.018277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152b64a-5323-428e-a139-451125f67958 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.018551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152b64a-5323-428e-a139-451125f67958 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.019262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152b64a-5323-428e-a139-451125f67958 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:36.019569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152b64a-5323-428e-a139-451125f67958 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:36.021052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152b64a-5323-428e-a139-451125f67958 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:36.035018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 247/488] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:36.045046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:36.241894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d229f-8f4e-44aa-b022-d34c990f55d3 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:36.273545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e73560-bb5d-408f-ae00-4bfb4a763fc9 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.274129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e73560-bb5d-408f-ae00-4bfb4a763fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.274129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e73560-bb5d-408f-ae00-4bfb4a763fc9 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.275833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 248/489] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:36.281813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.282134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.282549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.316352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:36.320035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.320880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.321349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.330111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:36.331751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:36.334744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:36.337517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c72fe4f3-3138-4cb9-88a9-2c9a2111ee0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:36.355876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:36.357186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:36.358090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 242/490] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:04:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:36.359074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec59d6-3561-45f1-b1f4-b5576579e974 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.359074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec59d6-3561-45f1-b1f4-b5576579e974 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.359289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec59d6-3561-45f1-b1f4-b5576579e974 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.360835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 243/491] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:36.361583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec59d6-3561-45f1-b1f4-b5576579e974 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:36.376983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:36.402654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:36.420069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 249/492] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:36.421328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:36.478968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec59d6-3561-45f1-b1f4-b5576579e974 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:36.524158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83010e3e-a1bc-4a77-b5af-46390582eb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:36.524707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010e3e-a1bc-4a77-b5af-46390582eb76 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.524985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010e3e-a1bc-4a77-b5af-46390582eb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.525257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010e3e-a1bc-4a77-b5af-46390582eb76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.537749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83010e3e-a1bc-4a77-b5af-46390582eb76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-83010e3e-a1bc-4a77-b5af-46390582eb76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:36.544186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:36.550893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 244/493] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:36.551925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.552881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.552881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.571271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:36.572750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:36.575669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ad77a-0a40-4f57-9a2d-03534515136c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:36.629717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:36.630808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.631565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.632299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.633577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:36.634068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:36.636874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:36.665495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 245/494] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:36.666617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:36.693441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 250/495] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:04:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:36.699482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:36.703901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:36.820291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:36.907060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a04171-9f22-4d4e-a046-15bd799b59a1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:36.928193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71550a97-2467-4727-a8e2-975360b82915 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:36.963133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:36.963726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.964046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.964324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.965039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:36.965366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:36.968893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:36.974893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:36.975540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:36.976365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:36.976646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:36.979097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 246/496] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:36.979952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.077449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.079751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.090282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.107516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322646d-d7e4-4c1d-ada5-7cf410841148 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.114642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.145892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.146539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.146819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.147415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.160326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:37.165981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:37.167325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 247/497] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:37.168230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.213227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 251/498] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:37.214609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.292763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f74041-f8e5-4bad-999c-e39a031b5a02 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.324773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65dee6-07dd-40b5-a3c1-49879f327561 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.332948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.333507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.333811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.334085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.345477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:37.350965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:37.352145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 248/499] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:37.352997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.359236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.363472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.369424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.370068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.370390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.375219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.382973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.384567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:37.401232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:37.402247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:37.403148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:37.403148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:37.403148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:37.403148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 252/500] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:37.403737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.470682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbccc83-250f-4be1-a990-b89831abca26 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.509267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.509880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.510160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.510427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.511094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:37.511392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.512832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33358bc7-04b8-48e9-b0e2-f370b9aa8714 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.519594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab51803-cb90-4db7-8322-65fa8e7e53db None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.521227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 249/501] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:37.522508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e339dd2-06b1-4312-b401-0b6b545b031f None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.522794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e339dd2-06b1-4312-b401-0b6b545b031f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.523116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e339dd2-06b1-4312-b401-0b6b545b031f None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.525129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 250/502] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:37.525985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e339dd2-06b1-4312-b401-0b6b545b031f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.558873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.559526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.559806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.560106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.572599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.574068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:37.590242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:37.591241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:37.592126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:37.592126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:37.592126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:37.592126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 253/503] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:37.592126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.608633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.626556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.647912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e339dd2-06b1-4312-b401-0b6b545b031f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.684643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.685254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.685522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.685791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.686452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:37.686749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.688245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.698918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 251/504] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:37.704308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.719432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407641-2503-4b46-bf5b-9fc41d806156 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.762130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.763142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.763421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.763694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.764465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:37.764764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.766276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b62f3-c777-4689-8ee0-2007d6397f62 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.781334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 254/505] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:37.782968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.783309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.783640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.804655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:37.806050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:37.833662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45285fd3-f23c-4c4e-8288-001dfcfa50a6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:37.867796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:37.873850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:37.874821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:37.875263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:37.875694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:37.891810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:37.893333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:37.898478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 255/506] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:37.899534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:37.909248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:37.910735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:37.911596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:37.911596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:37.911596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:37.916055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 252/507] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:37.917189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:37.938253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.041207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade599e-c622-4bae-9421-edb301edf924 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.065868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547ac14-c098-45f2-8fa4-676afa15be02 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.096216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.097216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.097688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.098137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.099144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:38.099667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.102823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.105578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.106406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.106949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.107255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.128031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.128031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.128529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 256/508] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:38.129838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:38.152245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:38.153472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:38.154359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:38.154359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:38.154359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:38.154359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 253/509] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:38.154646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.214906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.244069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.266025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6cf21-77f5-4cb1-a6d3-4c32b0e2df28 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.290119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebb661-7aff-4694-92f0-1ec54731f63b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.305078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.306151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.306852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.307200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.315167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.316711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.329812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:38.330323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:38.332619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:04:38.334367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:38.335799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.335799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.336033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 257/510] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:38.336033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.336530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.337710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:38.339694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.341461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.354864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 254/511] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:38.355710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.455525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c525b22-1c26-4d6c-b26f-8450fdfcf378 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.463548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.471494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e696d-96cd-405b-9cb0-aee96b21b604 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.498871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.499612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.499966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.500243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.510932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.512557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.516089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.516897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.517637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.518037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.518421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.519190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:38.519606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.521347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:38.529441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:38.530315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:38.531199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 258/512] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:38.531839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.544369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 255/513] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:38.545402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.656204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4bd0a-7deb-4cc7-b15b-258a4a8afefe None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.673414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2975b6-ae04-4a9c-8091-3a5f258ff662 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.693285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.694473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.694679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.694952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.710496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.711157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:38.725894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:38.727099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:38.728495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:38.728495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:38.728495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:38.728660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.728660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.728840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 259/514] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:38.728910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.728910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.729040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.729115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:38.729115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.730268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.776557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 256/515] 10.222.0.65 () {56 vars in 1137 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:38.778851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.798787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.802243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:38.857569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307614e6-d87d-42ad-aa81-632c03f71500 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.895811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.896509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.896790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.897139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.905375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.906899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:38.922176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829737f-da18-4627-b1ce-c4cee48c7e40 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:38.927215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:38.928889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:38.929950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 260/516] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:38.931211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:38.978699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:38.979230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:38.979509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:38.979781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:38.980717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:38.981022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:38.983595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.005351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 257/517] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:39.006210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.068726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3a480-e7ea-4e65-8c23-a5b8b440bd9d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.084403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.097035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.109386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.110070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.110375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.110673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.120432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.122293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.129937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857b315-c39f-4275-82e2-ba393ff63e26 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:39.136467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:39.137388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:39.138343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 261/518] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:39.139323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.170628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.171278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.171584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.171900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.172712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:39.174620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.179770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.192750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 258/519] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:39.193640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.262842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ed819-1afc-416f-9427-0a1ded98971b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.308537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.309159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.309448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.309767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.333712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f300278-2fc2-4e5c-a718-dfd5e72c1138 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.361227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.377044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.382528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.383179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.383485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.383763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.384575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:39.384899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.386493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.390583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.392141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.408348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 259/520] 10.222.0.65 () {56 vars in 1137 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:39.409264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.524430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 262/521] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:39.525384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.535945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21841-6c3d-44f0-88b7-f840d5c5ccd6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.573932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.574556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.574838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.575109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.576048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3e5014eab4c7479f9ee493e31c91b3f4) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:39.576369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.577879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.584067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 260/522] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/role_assignments?user.id=3e5014eab4c7479f9ee493e31c91b3f4&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:39.585439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.615105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.660233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.667082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d07a-1b58-4c70-988d-373480cbb832 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.717232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.718134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.718584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.718860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.719633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:39.719993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.721618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.730549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761049f-876c-4a86-9a4d-28a51134164b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.755877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 263/523] 10.222.0.65 () {56 vars in 1139 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:39.755877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:39.795803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.798988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.799337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.800369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:39.802943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:39.875588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1ea1e-e195-4412-bc71-add104d9e9ed None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:39.910680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:39.918824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:39.919499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:39.919786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:39.920194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:39.928424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 261/524] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:04:39 2022] POST /identity/v3/users => generated 247 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:04:39.930166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.034524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.036907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.099319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa8ff4-06f1-4d79-93bd-eb6be24a62a2 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:40.135623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:40.146148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:40.146797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:40.147117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:40.147398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:40.165722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.165970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:40.191760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:40.194832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:40.195707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:40.195707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:40.195707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:40.195707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 262/525] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:40.196900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.206723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 264/526] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:40.208468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.263424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:40.327989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173aedc-f23d-43e1-8033-f9bb628a6b82 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:40.362576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18681e0-7fa1-4a0d-9f01-b2d68c42d099 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:40.394955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:40.395966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:40.396322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:40.397626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:40.405815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:40.406596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:40.407032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:40.407457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:40.408362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:40.408679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.410118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.425244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 265/527] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:40.426135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.450946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:40.514146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.516490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.561828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:40.574423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe9b18-3ddd-4a26-9ba4-e052a3814a65 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:40.617941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:40.618559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:40.618945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:40.619247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:40.633257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.635041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:40.654727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:40.658612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:40.662255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:40.662255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:40.662255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:40.662255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 266/528] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:40.662255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.676781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 263/529] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:40.676781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.792258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71f6fd-e5c2-43d3-ae6c-cd55536afce2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:40.827942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:40.846707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:40.847361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:40.847647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:40.847985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:40.856799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.858460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:40.875194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:40.876843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 267/530] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:40.878516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:40.887960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:40.911161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3096c543-4d1d-4fef-bb5a-5e1b7d035b93 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:40.957118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:40.958155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:40.958155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:40.958387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:40.958910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:40.959216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:40.961202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:40.981619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 264/531] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:40.982631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.065771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aba200-c6fd-4fbf-9dd8-7ff874cab5ff None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.120121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.120833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.121123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.121401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.126505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba27fa3-c84d-44f7-bb38-9b436ca91840 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.140079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.142266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.160894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:41.164651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:41.165598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:41.166499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:41.166499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:41.166499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:41.170758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 268/532] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:41.171741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.181491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.182437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.183526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.183526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.184299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:41.184848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.186720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.208046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 265/533] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:41.209790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.254873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:41.311579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb82efc-f01f-43d1-b615-e003cb97df2f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.353266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.354969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.355506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.356026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.357676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e05e9-50aa-4439-ac7e-24f9ceadb235 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.365083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.368130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:41.383825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:41.384846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:41.385771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:41.388435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 269/534] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:41.390023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.402886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.403521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.403805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.404149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.404817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:41.405127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.406568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.424864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 266/535] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:41.426075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.480432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:41.523443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0860db3-6fe0-41fb-9d2e-e398173aead7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.542065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:41.548189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185f546-1f49-445e-ab8b-a244b3661704 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.568814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.569902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.570431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.570922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.571967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3fb180119c3a4f07933f7d24f4d22121) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:41.572581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.574487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.582517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 270/536] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/role_assignments?user.id=3fb180119c3a4f07933f7d24f4d22121&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:41.584076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.586666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.587297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.587579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.587855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.588593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:41.588901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.590670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.626554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 267/537] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:41.632657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.720913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a6a0a-c199-42a6-8524-79ee7aad60cd None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.782126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.782784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.783063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.783436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.792471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.794057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-936c935c-a550-4f79-b718-a3e728b054d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:41.811139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:41.812587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:41.813487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 271/538] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:41.814310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b869359-aa4a-4896-b510-a43d2155ca83 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.814551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b869359-aa4a-4896-b510-a43d2155ca83 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.814834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b869359-aa4a-4896-b510-a43d2155ca83 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.821230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 272/539] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:41.821550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b869359-aa4a-4896-b510-a43d2155ca83 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:41.835428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53671429-8374-4430-9800-9a0c7ee608ac None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:41.837729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:41.889336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:41.901965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:41.903040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.903547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.904153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.904932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=6ed41f2207eb4938b26f287a025f4bcc) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:41.905249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:41.906924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2adac-0661-4c82-ad3d-dea64b37bc8c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:41.928099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 268/540] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/role_assignments?user.id=6ed41f2207eb4938b26f287a025f4bcc&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:41.930113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006597af-a1ec-41da-9f41-637f09c0bfff None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:41.930465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006597af-a1ec-41da-9f41-637f09c0bfff None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:41.931016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006597af-a1ec-41da-9f41-637f09c0bfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:41.973090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-006597af-a1ec-41da-9f41-637f09c0bfff None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:41.974756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-006597af-a1ec-41da-9f41-637f09c0bfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:41.992774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b869359-aa4a-4896-b510-a43d2155ca83 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:42.092406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:42.093445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:42.093926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:42.095616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:42.096992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:42.097555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:42.104727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:42.171498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 273/541] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:42.184053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:42.189511 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 269/542] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:42.349364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:42.360584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda392f-6987-4406-a6e6-b2d1f230af01 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:42.411054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:42.411709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:42.412129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:42.413289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:42.413964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=73aebcec5ecc40ea9897fc7450c1cd0c) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:42.414278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:42.416183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e993a-c839-4100-9bf0-474c18e6a90d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:42.424535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 274/543] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:42 2022] GET /identity/v3/role_assignments?user.id=73aebcec5ecc40ea9897fc7450c1cd0c&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:42.607003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c35f-87c3-4405-b2ab-1da4bf15abe4 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:42.607460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c35f-87c3-4405-b2ab-1da4bf15abe4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:42.607460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c35f-87c3-4405-b2ab-1da4bf15abe4 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:42.609310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 270/544] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:04:42.615905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:42.616245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:42.616417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:42.641960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:42.643204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:42.679166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663c35f-87c3-4405-b2ab-1da4bf15abe4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:42.776400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 275/545] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:42.853177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:42.863678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663c35f-87c3-4405-b2ab-1da4bf15abe4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:42.928936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:42.930078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:42.930554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:42.931034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:42.961438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:42.970973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:42.971920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:42.972720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 271/546] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:42.983726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:43.185374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:43.204149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:43.221375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d821d2-37b3-4a1d-9da1-f4606f01ad0a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:43.312539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:43.313166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:43.313446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:43.313727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:43.314382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:43.314680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:43.316121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:43.348111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 276/547] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:43.352058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:43.444529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552d4a6-458b-4f9b-8770-acfd8f18e6d4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:43.530763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:43.531794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:43.532367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:43.532858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:43.548434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:43.559574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4baea-8e83-43a7-815a-443737614561 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:43.560639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:43.562874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:43.564147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:43.569112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 272/548] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:43.577381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:43.609004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:43.609659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:43.609944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:43.610215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:43.613468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:43.628727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:43.630525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:43.660700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:43.663757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:43.665614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:43.665614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:43.665614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:43.665614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 277/549] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:43.668512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:43.680703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:43.801363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b5a9f-804c-430d-90a9-9363d5b0c6b3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:43.819853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d798a-98c3-4f03-9b72-f688623d5d16 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:43.868134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:43.869290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:43.870317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:43.870801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:43.871561 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:43.871875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:43.873452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:43.894079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 278/550] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:43.901579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:43.922833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:43.924172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:43.924649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:43.925283 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:43.926176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:43.926713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:43.929167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:43.953644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 273/551] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:04:43.959904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:44.008460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:44.054940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d32f5-2203-455d-81d9-10cb8a196188 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:44.160003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c93011-996f-420d-931b-e80f43296aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:44.161065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c93011-996f-420d-931b-e80f43296aef None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:44.161556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c93011-996f-420d-931b-e80f43296aef None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:44.162016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c93011-996f-420d-931b-e80f43296aef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:44.195196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c93011-996f-420d-931b-e80f43296aef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:44.197382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c93011-996f-420d-931b-e80f43296aef None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:44.228294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-06c93011-996f-420d-931b-e80f43296aef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:44.229152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:44.230267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:44.232751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 279/552] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:04:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:44.240815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c93011-996f-420d-931b-e80f43296aef None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:44.278478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1a21e0-6c25-4cd8-8173-f8c192c86609 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:44.339595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:44.340956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:44.341673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:44.342947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:44.361913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:44.363055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c3d597c8-0b3c-4a7f-a0f8-ec7a691aab7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:44.386889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:44.387914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:44.389887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:44.389887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:44.389887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:44.389887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 274/553] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:44.390239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39f16-fa48-43ae-bba4-cc5d7adff9a9 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:44.390488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39f16-fa48-43ae-bba4-cc5d7adff9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:44.390769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39f16-fa48-43ae-bba4-cc5d7adff9a9 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:44.392623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 275/554] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:44.400171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:44.401266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:44.401927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:44.433824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:44.436041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:44.469040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c93011-996f-420d-931b-e80f43296aef None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:44.521152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:44.534217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:44.535366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:44.535987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:44.536434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:44.541091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:44.541091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:44.541091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:44.574668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 276/555] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:04:44.575864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:44.579294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 280/556] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:04:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:44.581461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:44.677342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:44.722166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619c46b-3050-49e2-a6e3-fe8c5a2570cb None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:44.778251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:44.779577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:44.780196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:44.780684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:44.781617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:44.782130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:44.783799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:44.799201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 277/557] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:44.807134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:44.911344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41ac94-b201-4c78-82b8-ec014f4123c1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.016199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc6fed-8087-4995-9835-b53cf21bd437 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.036702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.038377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.038842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.039409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.049494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:45.068109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.093594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.093594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.093594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.093594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:45.093594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:45.094626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:45.094626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-1081280d-d5e0-422e-9aae-1ebc8187cf67 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:45.107802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:45.108941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:04:45.109900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:45.111359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 278/558] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:04:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:45.116743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bc1af-b436-4041-be4a-684331971d66 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.117250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bc1af-b436-4041-be4a-684331971d66 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.117720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bc1af-b436-4041-be4a-684331971d66 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4251aa85-4540-48e0-9427-4d9e2e16b4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:45.118699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:45.119829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:45.120943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:45.120943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:45.120943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:45.124508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 281/560] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 544 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:45.124747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 279/560] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:45 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:45.128851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bc1af-b436-4041-be4a-684331971d66 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.136319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.136934 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.137508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.172007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:45.173421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:45.213129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:45.264111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 282/561] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:45.265620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.310124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bc1af-b436-4041-be4a-684331971d66 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.358296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.359257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.359631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.360135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.361373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:45.361872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:45.364330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:45.400401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 280/562] 10.222.0.65 () {56 vars in 1135 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:45.401368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.429863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549aadd-d8e2-4403-97ba-87a739dc4f79 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.459046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:45.489489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.491364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.491864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.492318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.493127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:45.493668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:45.495691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:45.519949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 283/563] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:45.519949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.549041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321a630-3202-4af9-9d41-d6545ee60264 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.569972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:45.614868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.615982 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.616474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.616998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.637060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:45.646102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:45.647661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:45.648647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 281/564] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:45.650542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.684024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d16a2-3929-49b9-8cc9-ea1e39cb125b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.722766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.723482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.725774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.725774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.763113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:45.767092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023f0d-b319-4c93-b0bd-a20952894ded None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:45.803762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:45.804453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:45.804732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:45.805006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:45.808840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:45.810422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:45.817564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:45.819192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:45.832723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:45.834962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:45.836813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:45.839132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 282/565] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:45.840339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.844173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:45.955490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 284/566] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:45.956463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:45.979859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e151c3-7514-47f6-a554-93e783fca60d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.024681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6461-21c1-4e0a-adf6-ce925a918b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.025500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6461-21c1-4e0a-adf6-ce925a918b8f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.025893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6461-21c1-4e0a-adf6-ce925a918b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.026237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6461-21c1-4e0a-adf6-ce925a918b8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.040477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6461-21c1-4e0a-adf6-ce925a918b8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-a5fb6461-21c1-4e0a-adf6-ce925a918b8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:46.046463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:46.047562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:46.048542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 283/567] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:46.050619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aaa3f-d455-4346-8add-cd774633a31b None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.051088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aaa3f-d455-4346-8add-cd774633a31b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.051531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aaa3f-d455-4346-8add-cd774633a31b None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.054411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 284/568] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:46.055757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aaa3f-d455-4346-8add-cd774633a31b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.086343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:46.105679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8f942-a4e9-472f-9b36-5cdcf0a38f6d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.140687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:46.152675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.153305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.153588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.153906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.157178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:46.157178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.157178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.167013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 285/569] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:46.167911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.198097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aaa3f-d455-4346-8add-cd774633a31b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.238292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.238951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.239231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.239498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.240758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:46.241070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.242563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f029ea-6e44-4f99-97b5-0442b032dcc7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.259969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 285/570] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:46.259969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.259969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.259969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.278022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:46.279255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:46.289944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca257e-0b64-45f8-a49e-9ca35a52c342 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.334993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.335639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.335944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.336235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.348119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.350345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.359066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 286/571] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:46.360018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.371624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:46.377058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:46.378874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:46.381238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:46.381238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:46.381238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:46.383079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 286/572] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:46.384139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.459270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:46.528949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749ea60-ef2c-4203-a161-ebd96cb42d9b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.543488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adfe70-f315-4782-b9bb-9a3d47379ca2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.588440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.589085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.589416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.589707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.592106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.593138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.593629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.594116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.595317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:46.595866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.598726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.601983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.603774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.619563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 287/573] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:46.619563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:46.620456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:46.621436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:46.622331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:46.622331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:46.622331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:46.626426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 287/574] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:46.627173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.708654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:46.734044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:46.742283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc39f4-9759-491d-9d04-ad9700cf4d1b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.752845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d616-db01-4176-91cb-7b621573f233 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.780846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.781645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.782022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.782387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.783242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:46.783628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.785393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.790732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.791507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.791905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.792286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3e5014eab4c7479f9ee493e31c91b3f4/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.798135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 288/575] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:46.799037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.808725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=3e5014eab4c7479f9ee493e31c91b3f4, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.810085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.901432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998b140-0d20-4a26-8932-55ea51bb387d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:46.938791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 288/576] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:04:46 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3e5014eab4c7479f9ee493e31c91b3f4/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:46.939618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.941765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:46.942423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:46.946304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:46.946304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:46.957030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:46.959263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:46.977873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:46.979367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:46.981367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:46.981367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:46.981367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:46.986311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 289/577] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:46.987279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:46.994647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.127750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f1804-5164-4f0c-982d-58357eb4f136 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.152155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.166089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ff79c-2c34-4fda-b1cc-a30de8a4aef8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.172231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.172861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.173143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.173420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.174565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:47.174856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.176667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.194446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 289/578] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:47.195338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.212266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.213126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.213490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.213856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.228692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:47.235332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:47.236432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:47.237281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 290/579] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:47.237997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.323358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.330022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d1772-2c25-4791-8851-c0f9adc140fe None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.369359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.370451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.370948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.371395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.373668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf62e9-4eb9-4cbf-ae56-90da4e69dfba None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.382188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.384412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:47.397083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:47.399064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 290/580] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:47.400310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.428604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.429142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.429409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.429719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.430370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:47.431398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.432209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.448216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 291/581] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:47.449283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.471229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.545170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a44b3-ad9d-4049-8001-78edcc757cda None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.574573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880673f-e2a5-4930-9f63-abf86e66c8b6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.582193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.582834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.583112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.583436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.592136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.593612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.604045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:47.606952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:47.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 291/582] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:47.611647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.626700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.627401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.627683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.628025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.628709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:47.629069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.630509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.647933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 292/583] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:47.647933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.746682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.750359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42cae1-aec6-442b-b5cd-d595c1d04c0c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.781119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfe140-d891-43e7-a658-19e6c6d1d2e1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:47.791776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.792607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.793049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.793433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.794258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:47.794672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.796421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.818430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 292/584] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:47.818430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.836666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:47.837746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:47.838230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:47.838720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:47.850258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:47.851797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:47.869887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:47.871173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 293/585] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:47.877159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:47.939188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:47.968166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fdf84-46d8-4cdf-8412-50defd40eaa3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.016046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.016576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.016856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.017142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.017884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:48.018506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.019909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.027130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33e881-a89b-4b1f-a3e2-53eb9c9052c1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.043178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 293/586] 10.222.0.65 () {56 vars in 1137 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:48.044087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.067923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:48.074361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.075016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.075301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.075943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.089729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.091555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:48.111736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:48.112893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:48.114069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:48.114069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:48.114069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:48.119581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 294/587] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:48.119581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.180587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700340cc-ab5b-4b38-8c58-21de34f4c6ec None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.223840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.224510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.224787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.225066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.225793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=fb437e7c06504d0aa7891269ed02f9e6) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:48.226126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.227582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.239941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 294/588] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/role_assignments?user.id=fb437e7c06504d0aa7891269ed02f9e6&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:48.239941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.284072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:48.293528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c02be-594f-4d8b-b589-22dcae9fc32d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.338915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.339785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.339785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.340174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.340928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:48.341233 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.342886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.349427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:48.369734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 295/589] 10.222.0.65 () {56 vars in 1137 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:48.372748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.377430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f43b2f-ca60-414f-99ef-3431d02e6432 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.439986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.441424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.442532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.444025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.471957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.476048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.496172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e01f0e-fd42-4eb0-aaee-173498c30a26 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:48.518976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:48.520806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:48.523202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:48.523202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:48.523202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:48.527116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 295/590] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:48.528934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.553532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.556291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.557229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.558330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.560413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:48.561317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.565367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.597961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 296/591] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:48.600038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.637367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:48.700439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756eed-a7d9-4566-98b0-ff0d355c34a7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.744273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.745508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.746126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.746781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.749756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:48.763250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.765904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.772335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc5c63-7b9e-4552-a3ea-c478007acc03 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:48.787057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:48.788121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:48.789025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:48.789025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:48.789025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:48.797921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 296/592] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:48.799920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.812298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:48.812973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:48.813086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:48.813424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:48.814006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:48.814319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:48.815693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:48.829836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 297/593] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:48.830688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:48.945779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:48.946692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43730c9-966c-46ce-aacd-8e4c2c549dd4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:48.971485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ee790-7bf4-40fe-bc0f-fa9a092fceee None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.004433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.006206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.006994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.007869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.018399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.019387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.019860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.020366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.021526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:49.022051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.024609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.028512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.032011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.050135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 298/594] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:49.050135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:49.064471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:49.066901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 297/595] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:49.068841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.168187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:49.199723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64167d5-ba35-4054-94ef-fe371aa1c38f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.239136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3dea7-b4e0-46ab-a897-da148d84babf None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.246617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.248320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.248320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.248320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.252204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:49.257677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.259505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:49.275466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:49.276666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:49.277795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 299/596] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:49.278421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.294726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.295647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.295955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.296232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.304760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.306282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b70cc342-e5ee-44af-b6f1-142271868fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:49.321870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:49.322820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 298/597] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f4a9-b358-4331-b78e-40b914e59e51 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.329412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f4a9-b358-4331-b78e-40b914e59e51 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.330707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f4a9-b358-4331-b78e-40b914e59e51 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.332034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 299/598] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:49.333156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f4a9-b358-4331-b78e-40b914e59e51 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.421974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4b4c8-3ead-496a-85cd-ae74512ebc5e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.463174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.464090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.464376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.464657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.465394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:49.465704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.467240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.486718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f4a9-b358-4331-b78e-40b914e59e51 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.500403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 300/599] 10.222.0.65 () {56 vars in 1139 bytes} [Mon Aug 29 21:04:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:49.500403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.502927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:49.546815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.547575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.547860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.548170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.548936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:49.549241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.550735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098dea3-30ee-4039-bc87-8eed5ff5df49 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.564053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:49.570411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 300/600] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:49.570411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.570411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.570411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.590592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:49.591863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:49.635114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643ec94-dc4f-445a-9f5d-95e8b4f9d16e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.682257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 301/601] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:49.683197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.693382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.694388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.694756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.695100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.695935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:49.696272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.698102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.713891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 301/602] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:49.714844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:49.835304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:49.844187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227b6b8-9ea2-4ff9-9f31-4e939a413fd1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.857015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6008a22-2c6a-4d3c-a237-14c74d79ee67 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:49.873078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:49.897683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.898897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.899448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.900065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.901774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:49.902706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:49.903221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:49.903674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3fb180119c3a4f07933f7d24f4d22121/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:49.915906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.918063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.923850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=3fb180119c3a4f07933f7d24f4d22121, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:49.925675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:49.937610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:49.938975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:49.940061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:49.940061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:49.940061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:49.940061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 302/603] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:49.940061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:50.076333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b70e18-2d05-41ab-b1ff-5047bbc25567 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:50.094167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 302/604] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:04:49 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3fb180119c3a4f07933f7d24f4d22121/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:50.105439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:50.122026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:50.122725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.123052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.123346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/6ed41f2207eb4938b26f287a025f4bcc/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.145390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=6ed41f2207eb4938b26f287a025f4bcc, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:50.147367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:50.267617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e9b7-f9f0-4c81-9e7a-81a8410f7a10 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:50.322358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:50.336180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 303/605] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:04:49 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/6ed41f2207eb4938b26f287a025f4bcc/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:50.337395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:50.337647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:50.338494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.338988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.339548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.340696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:50.341207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:50.344122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:50.375546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 303/606] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:50.382778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:50.446824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:50.559133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a75c5-93cd-4734-a613-afcd146259bf None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:50.585509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383309a-4143-4cfe-91f4-daf0e021a0ee None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:50.606097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56ee83-8843-4080-b084-20891f4d80c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:50.607138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ee83-8843-4080-b084-20891f4d80c6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.607620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ee83-8843-4080-b084-20891f4d80c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.608140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ee83-8843-4080-b084-20891f4d80c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.624572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56ee83-8843-4080-b084-20891f4d80c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6e56ee83-8843-4080-b084-20891f4d80c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:50.632230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:50.633434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:50.634539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:50.635445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:50.635445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.635445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.635445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.639986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 304/607] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:50.639986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84579462-78b4-4603-8f30-b0d893c97c92 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.639986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84579462-78b4-4603-8f30-b0d893c97c92 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.639986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84579462-78b4-4603-8f30-b0d893c97c92 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.640905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 305/608] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:50.646316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84579462-78b4-4603-8f30-b0d893c97c92 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:50.650332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:50.651912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-6e98f6b0-b990-4289-b84c-a9fa9e09c87c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:50.667243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:50.671462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 304/609] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.673733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.675266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.694195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:50.695409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:50.768322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:50.824308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 305/610] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:50.825399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:50.836481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84579462-78b4-4603-8f30-b0d893c97c92 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:50.860983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:50.895097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:50.897129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:50.897761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:50.898192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:50.899466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:50.900070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:50.902224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:50.923415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 306/611] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:50.926688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.056313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777ead4-e67f-42a3-b954-b22b234ee07f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.062272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb54013-ebda-4823-8e0e-81f9ba080a54 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.093082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:51.114656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:51.115729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:51.116256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:51.116845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:51.134050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:51.135957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:51.147873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:51.148509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:51.148837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:51.149116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:51.149858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:51.150214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:51.152181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:51.159657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:51.160727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:51.161665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:51.161665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:51.161665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:51.168129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 307/612] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:04:51.172260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.176074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 306/613] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:51.176936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.298234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10932-5d72-49b7-b16a-b4518bcd9734 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.328943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:51.343814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:51.344870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:51.345392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:51.345881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:51.346828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:51.347368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:51.349259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:51.370741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 308/614] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:51.372689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.382355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834d4aa-0969-434a-b979-94474aca516b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.411926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:51.452975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:51.454051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:51.454731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:51.455176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:51.477329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:51.488542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:51.490084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:51.490915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:51.492145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 307/615] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:51.492967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.507527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c9a28-06d6-4776-89cf-74c4b0e5a8d1 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.546655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:51.547688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:51.548265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:51.548752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/73aebcec5ecc40ea9897fc7450c1cd0c/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:51.566074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=73aebcec5ecc40ea9897fc7450c1cd0c, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:51.568967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:51.679915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 309/616] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:04:51 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/73aebcec5ecc40ea9897fc7450c1cd0c/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:51.681449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.684394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc60b3-9a7f-4fc0-b29d-4e755611a22f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.730521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:51.731857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:51.732641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:51.733153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:51.737528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:51.749891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:51.751931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:51.767918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:51.769158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:51.770204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:51.778708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 308/617] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:04:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:51.783130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:51.914291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:51.943170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ccd2c-faa0-4aed-a443-8ce8a0e97484 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.948423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9d9e0-dea6-41bf-a98e-d66049c4cb22 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:51.999463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:52.001369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.002057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.002802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.004564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:52.005261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:52.006198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:52.006972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.007376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.007812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.008653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:52.009069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:52.009525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:52.010784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:52.028945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 310/618] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:52.038617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:52.059152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 309/619] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:04:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:52.066002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:52.172032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:52.289942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3f6c6-3e5f-449a-82be-3f91a058678a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:52.295820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a66908-87c4-422b-978c-0ca9e99ed7a7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:52.356056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1754ec-36a5-4687-9911-9676cd3056c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:52.357384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1754ec-36a5-4687-9911-9676cd3056c9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.357676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1754ec-36a5-4687-9911-9676cd3056c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.357960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1754ec-36a5-4687-9911-9676cd3056c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.379576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:52.381005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.381405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.381748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.394039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:52.396172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:52.399280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:52.413417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:52.429560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:52.430541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:52.430541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:52.430541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:52.430541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 311/620] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:52.430541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:52.508562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1754ec-36a5-4687-9911-9676cd3056c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:52.510090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1754ec-36a5-4687-9911-9676cd3056c9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:52.594734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39382ebc-324a-426a-b347-47c7fc0f4a18 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:52.600852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:52.660073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 310/621] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:52.661609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d8a06-3e92-4801-b7bb-a1b1f2606bdd None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.661962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d8a06-3e92-4801-b7bb-a1b1f2606bdd None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.662344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d8a06-3e92-4801-b7bb-a1b1f2606bdd None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.663487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:52.665075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.665669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.665952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 311/622] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:52.666371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.666856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d8a06-3e92-4801-b7bb-a1b1f2606bdd None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:52.667562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:52.668145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:52.669925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66391b97-914d-47e3-a022-517c8f0c066a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:52.689688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 312/623] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:52.692912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.693417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.694264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.725277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:52.728054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:52.812800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:52.830599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d8a06-3e92-4801-b7bb-a1b1f2606bdd None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:52.840056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 313/624] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:52.841232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:52.880402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:52.880960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:52.881247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:52.882140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:52.893756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:52.895351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:52.914217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:52.915251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:52.916186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:52.916186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:52.916186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:52.916769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 312/625] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:52.923228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:52.927405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:53.056433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2b1ad-970a-4c92-8fd3-a1e382ee974d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.078501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e6d7b-7167-45e6-a071-7ed7e911e102 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.102100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.102730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.103076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.103348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.104039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:53.104345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:53.106028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:53.120593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 313/626] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:53.126241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.167583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.168510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.168861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.169141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.192597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:53.194200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:53.200218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6e6b462f-8321-4541-b51c-a69e55e37a3c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:53.216264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:53.218738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 314/627] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:04:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfbc3a-f9f2-4177-9e9f-759f7f777cbb None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.221852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfbc3a-f9f2-4177-9e9f-759f7f777cbb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.228468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfbc3a-f9f2-4177-9e9f-759f7f777cbb None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.228468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 315/628] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:53.228468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfbc3a-f9f2-4177-9e9f-759f7f777cbb None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.253761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e83bb-614e-47e7-a179-a3cf39230a5c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.302777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:53.304671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.305950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.306714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.307460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.317758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:53.319575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-d624c506-fb87-42f7-9fe3-5ac733d90717 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:53.340162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:53.341892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:53.343612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:53.345541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 314/629] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:53.352266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.352266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.352266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.384112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:53.386127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfbc3a-f9f2-4177-9e9f-759f7f777cbb None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.386536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:53.441745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.446492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.446492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.446492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.446492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:53.446492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:53.446492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:53.471484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 316/630] 10.222.0.65 () {56 vars in 1135 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:53.471484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.507487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 315/631] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:53.508968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.636346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:53.640518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfefed-afee-409f-a2c0-cbb60580b47f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.644981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:53.681736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.682428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.682704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.682984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.683963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:53.684090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:53.685475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:53.719288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3519ab8-5d9a-470a-b578-85a230a1769f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.719536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 317/632] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:53.719536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.757610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.761280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.766289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.767780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.784921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:53.800160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:53.801241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:53.802738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:53.807810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 316/633] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:53.810121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.849799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462b7c3-91ae-4b86-a79f-31ab3dffa21b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.894279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:53.894950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:53.895232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:53.895530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:53.908881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:53.910343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:53.924747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:53.925780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:53.927217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:53.927217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:53.927217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:53.931245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 318/634] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:53.932030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:53.951553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:53.968316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2b9b8-864a-4503-97ce-10cafa36d11f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:53.984974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:54.012731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.013475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.013824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.014152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.014855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3e5014eab4c7479f9ee493e31c91b3f4) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:54.015281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.016973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.023833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 317/635] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/role_assignments?user.id=3e5014eab4c7479f9ee493e31c91b3f4&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:54.029693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.072717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a21a99-e6e6-4ef8-a3f0-2fb9b4e2d6a7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.127564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.127564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.127564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.127564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.128034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:54.128329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.129753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.148850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 319/636] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:54.150130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.219799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950038ac-bbf0-46c1-8f33-5c75528b9b70 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.261653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:54.276667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.277383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.277856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.278325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.296153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:54.298186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04270c-92af-48c8-91ae-4ebe3dd27c4a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:54.302597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:54.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:54.304421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 318/637] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:54.305119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.350838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.351677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.352162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.352576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.353539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:54.353999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.356101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6c83f-45b7-4469-af6e-de580e1a00d8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.361592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:54.375227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 320/638] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:54.375227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa242f-539b-447f-bde1-443e595e4497 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.375227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa242f-539b-447f-bde1-443e595e4497 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.375227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa242f-539b-447f-bde1-443e595e4497 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.375227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 321/639] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:54.375227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa242f-539b-447f-bde1-443e595e4497 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.437199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273387bc-4775-4663-b55e-e4f61c7a73c8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.481726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.482426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.482760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.483086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.495772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.497370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.502372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa242f-539b-447f-bde1-443e595e4497 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:54.514496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:54.515470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:54.516429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:54.516429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:54.516429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:54.520352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 319/640] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:54.521638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.550779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.551401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.552236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.552715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.553219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:54.553772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.555501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7cbed-281d-435a-a003-86a962ccb6f9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.561804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:54.571869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 322/641] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:54.584195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.584195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.584195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.597761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:54.600142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:54.697773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:54.723910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 323/642] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:54.723910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.728505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d37963-7d59-4c02-ab7a-0ae47a6ef370 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.790902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.791793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.792305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.792771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.805312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.807849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:54.831012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:54.832125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:54.833096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:54.840423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 320/643] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:54.842140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:54.884712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998109d-8eca-4435-9bdf-625e4df73346 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:54.940028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:54.950557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:54.951417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:54.951964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:54.952495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:54.953751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:54.954254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:54.957055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:54.984289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 324/644] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:54.986222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.007615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef95d2-bb0b-4cfc-8b1a-d5631615261f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.057681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.058708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.059186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.059645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.075810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:55.078280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:55.102330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:55.103406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:55.104516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:55.104516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:55.104516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:55.112580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 321/645] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:55.114674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.121224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:55.166576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68891a84-5acc-47ca-8330-04046441adf8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.212071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.212877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.213237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.213711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.214890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:55.215433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:55.217522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:55.237377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 325/646] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:55.238497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.272603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db7713-45ed-41cc-8820-40b9293969c1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.302466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:55.321357 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.322002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.322285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.322839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.333517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:55.336290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:55.352448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:55.353514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 322/647] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:55.354491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.375943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1da89-6a29-41c2-ab6b-778704c97f34 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.417727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.418403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.418682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.418951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.419750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:55.419972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:55.421426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:55.446157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 326/648] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:55.446157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.466545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:55.524560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbe862-d417-4493-ae7f-43c574628dd1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.567126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.568026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.568484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.570604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.588582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:55.597557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:55.599989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 323/649] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:55.601973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.623333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1ae5c-c3fc-4b4c-803b-c53434415814 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.645742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:55.678886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.679604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.679906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.680188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/fb437e7c06504d0aa7891269ed02f9e6/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.703592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=fb437e7c06504d0aa7891269ed02f9e6, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:55.705426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:55.746443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ffbf8-1e08-45e5-91b6-48248d0aa5c5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.771018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:55.783359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.784041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.784335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.784617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:55.785391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:55.785717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:55.787191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:55.806064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 324/650] 10.222.0.65 () {56 vars in 1137 bytes} [Mon Aug 29 21:04:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:55.806900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.841207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 327/651] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:04:55 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/fb437e7c06504d0aa7891269ed02f9e6/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:04:55.842706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:55.948291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87040c6-03ff-4f36-ab6e-0a7a03fe5036 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:55.997489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:55.998190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:55.998486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:55.998761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.001673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338943c5-7a03-48bf-b8f5-0e62118b9b39 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:56.008113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.009729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:56.023712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:56.024919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 21:04:56.026098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:56.026681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 325/652] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:56.027516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.083662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:56.084949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.085467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.086116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.087097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:56.087618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.090171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.092010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.103525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 328/653] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:56.104814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.144355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.181015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db161e56-5f57-4c2a-bdd2-ad3bfd5b34cc None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:56.222849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35aabc-dc17-49da-bd3b-086c68e9409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:56.223495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35aabc-dc17-49da-bd3b-086c68e9409d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.223784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35aabc-dc17-49da-bd3b-086c68e9409d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.224091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35aabc-dc17-49da-bd3b-086c68e9409d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.261770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b76da-5c93-444f-b517-a2e0a697c137 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:56.302269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:56.302918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.303243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.303534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.304301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:56.304603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.306124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.307222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35aabc-dc17-49da-bd3b-086c68e9409d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.309516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35aabc-dc17-49da-bd3b-086c68e9409d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.326154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 329/654] 10.222.0.65 () {56 vars in 1137 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:56.327006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.366482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.428959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.449294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de0094-706e-4dee-ad7d-bd57584cad56 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:56.466980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 326/655] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:56.466980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a5be-4313-4f32-9fea-5db98f5e214e None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.466980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a5be-4313-4f32-9fea-5db98f5e214e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.466980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1a5be-4313-4f32-9fea-5db98f5e214e None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.467402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 327/656] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:56.468314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1a5be-4313-4f32-9fea-5db98f5e214e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.495432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:56.496133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.496387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.496655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.507990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.509388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:56.528218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:56.529412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:56.530325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:56.530325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:56.530325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:56.530325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 330/657] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:56.530325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.604435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1a5be-4313-4f32-9fea-5db98f5e214e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:56.645282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:56.646250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.646788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.647331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.651874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a38a3-232e-45c5-9923-42654a624cf2 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:56.659936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.686373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.696796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:56.697851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.698170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.698445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.709755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.711122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b7804fef-93f6-4c89-bab9-27e208404ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:56.726341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:56.727519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:56.728945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:56.728945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:56.728945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:56.729155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:56.735102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 331/658] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:56.735310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:56.735425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:56.735425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:56.735425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:56.752919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:56.754148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:56.844095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 332/659] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:56.844095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.888162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 328/660] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:56.889190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:56.947020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:56.999771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8260e0-4d98-4fd8-ba66-4ca18acc2eab None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.021612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:57.044881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebfcfa-299c-4887-aa73-3c21860194c2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.049656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.050575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.051006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.051502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.052335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:57.052831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.055672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.075628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 333/661] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:57.076905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.101591 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.102198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.102484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.102790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.103977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:57.104416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.106079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.120145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 329/662] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:57.121305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.217276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a02f8a-a998-4543-bb21-9e3ab0e1eae3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.241807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:57.245394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641520-11c5-4e0c-bcb7-0f969a9f63de None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.261722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.262965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.263475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.263930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.276195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.277900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.286377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.287025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.287305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.287577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.295751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:57.296798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:57.297854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:57.298943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:57.298943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:57.298943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:57.299133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.300499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 334/663] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:57.301544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:57.318876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:57.320420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 21:04:57.321612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:57.326356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 330/664] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:57.326930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.339865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:57.422371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3fec2-93a4-4e9e-96cf-d5774b9aec83 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.445411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9aabd-98e4-4e5e-8980-1f9b2237c227 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.462550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.463316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.463616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.464038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.471540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.473323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.485570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:57.486312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:57.487378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:57.493812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:57.494360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.494360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.494517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 335/665] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:57.494607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.494689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.509662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:57.515673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:57.517687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 331/666] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:57.524564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.552279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:57.617562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:57.647255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ca324-141d-4180-80b3-ddc2c93b9253 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.661114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d519-561b-47bf-a6ca-6e22fb81e644 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.707147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.707961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.708251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.708527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.709189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:57.709543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.711082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.715577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.716396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.718514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.718514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.718514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:57.718514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.719590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.726779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 336/667] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:57.727861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.755474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 332/668] 10.222.0.65 () {56 vars in 1139 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:57.757192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.882854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:57.898466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f984c-d293-46dd-a78b-d645c59e2166 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.909756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746e659-2cb2-4445-96dc-3de0676c5a41 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:57.940312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.940938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.941215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.941495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.942382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:57.942800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.944288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.948176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:57.948955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:57.949208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:57.949672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:57.950448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:57.950762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:57.952178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:57.958505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 337/669] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:57.959369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.964910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 333/670] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:57.965162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:57.972465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.112683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dbc4f6-5deb-403b-baa5-3f089dd48f20 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.131759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0659f27-647d-4fa4-8dcb-1b889b77315c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.156800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.157443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.157723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.158043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.166171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.167662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.168093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.168708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.168986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.169251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.180916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7df67d2f-5064-4daa-9f84-5f594b32ce63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:58.181366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:58.182336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:58.183233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 338/671] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:58.184034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b2a8d-6656-439b-9f18-e652ba4a8a1e None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.184260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b2a8d-6656-439b-9f18-e652ba4a8a1e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.184567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b2a8d-6656-439b-9f18-e652ba4a8a1e None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.185872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 339/672] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:04:58.186675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b2a8d-6656-439b-9f18-e652ba4a8a1e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.255394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.256932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.278725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.319147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b2a8d-6656-439b-9f18-e652ba4a8a1e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.354424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.355061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.355595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.355867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.367311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.368647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 334/673] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:04:58.369358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.369509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:58.390965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:58.392046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:58.392961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:58.392961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:58.392961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:58.392961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 340/674] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:58.393184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.426815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.491870 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81b71b-b186-43ea-84d2-3c54bef9b616 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.508906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d21e-7ec6-46cb-adf2-e2aca4debad5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.529211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.532770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.533387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.533665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.533987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.534663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3fb180119c3a4f07933f7d24f4d22121) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:58.534967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.537095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959cafb-1041-4f1f-b56b-cf87538a9297 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.543298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 335/675] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/role_assignments?user.id=3fb180119c3a4f07933f7d24f4d22121&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:58.546588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.546588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.546588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.551552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.552344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.552624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.552900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.553624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:58.553936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.555500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.568572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:04:58.570538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:04:58.573445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 341/676] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:58.577355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.682267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 336/677] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:04:58.683370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.740320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.767366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a032360-b6cb-4aa3-a8bb-211b812997b9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.816313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.817872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.818304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.818773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.829740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2226d-b39e-4185-bed3-c925f0e948ce None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:58.831396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.834009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:04:58.860331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:58.861350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:04:58.862234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:04:58.862234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:04:58.862234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:58.867611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 342/678] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:58.868862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.885326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:58.885829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:58.886148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:58.886374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:58.887094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:58.887453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:58.889093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:58.905082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 337/679] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:58.905962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:58.912784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:58.997588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc20cd2-da16-42c6-95b4-4a0e99234ff4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.046947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.047879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.048190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.048485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.049379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=6ed41f2207eb4938b26f287a025f4bcc) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:59.049754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.055937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.067938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 343/680] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/role_assignments?user.id=6ed41f2207eb4938b26f287a025f4bcc&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:59.067938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.068369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1daa4f-6370-4c3f-9f27-98a00a972b67 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.069166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:59.109849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.110476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.111380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.111663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.112372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:59.112677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.114963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.134254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 338/681] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:59.135744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.237474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:59.246736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb62aac-318c-43f2-9215-6333b3cdccae None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.291868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bad9d-f249-452a-83c2-242f670b0fa4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.304435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.305090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.305369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.305641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.315006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.318063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:04:59.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:59.331530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:04:59.332635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:59.335967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 344/682] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:59.337054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.338774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.339431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.339714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.340094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.353799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:04:59.359425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:59.362056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 339/683] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:59.365588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.397664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:59.485648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d4755-8904-4d07-9221-5226fa3060a3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.528368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44b330-aba2-426b-a7f8-1e9c244f6c6c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.545244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.546414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.547698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.548512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.563829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.567235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:04:59.584676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:04:59.585676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:04:59.586571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:04:59.590671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 345/684] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:04:59.591679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.602946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.603735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.604847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.604847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.606517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:59.607148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.609616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.628582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:59.642216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 340/685] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:59.642216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.748252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:04:59.763133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef4b7d-4af7-4396-a0ad-24ed6105ea59 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.807546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1fe36-e3f1-4a67-a58d-03e8e5cfd8f9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:04:59.823258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.824109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.824690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.825045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.826679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:59.827030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.829149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.849330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 346/686] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:59.849330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.872792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:04:59.874576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:04:59.875092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:04:59.875543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:04:59.876992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:04:59.877712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:04:59.880735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:04:59.908845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 341/687] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:04:59.911528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:04:59.976764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:00.015585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490b953-2b18-4aef-94cb-a58ae3a22358 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:00.076118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:00.077100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.077630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.078173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.097268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e5be7-87f5-442f-9387-0815ace88d58 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:00.149534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:00.157366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:00.158446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.161903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.161903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.175742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:00.178249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:00.178372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:00.182404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:00.201241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:00.202730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:00.204221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:00.204221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:00.204221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:00.204221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 342/688] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:00.205253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:00.308515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:00.352190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 347/689] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:05:00.355707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:00.370252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516bc6-f534-483d-8763-6af8a7d0b7d0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:00.429817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:00.430820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.431245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.432214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.432976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=73aebcec5ecc40ea9897fc7450c1cd0c) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:00.433571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:00.436681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:00.445213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 343/690] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:00 2022] GET /identity/v3/role_assignments?user.id=73aebcec5ecc40ea9897fc7450c1cd0c&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:00.446210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:00.525432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a9d3c-0234-45a0-84b7-44c7c26c81f2 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:00.557835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:00.591443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:00.592132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.592414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.592686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.593368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:00.594393 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:00.595933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:00.632328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 348/691] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:00.638280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:00.688323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae7b5b-11a1-4bfc-9d3e-83bc31957ed2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:00.706371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:00.743203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:00.745276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.746539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.747205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.774732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:00.776919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:00.797053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:00.798521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:00.799428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:00.799428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:00.799428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:00.801575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 344/692] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:00.805310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:00.821924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667ccff-d508-4278-aff7-f15a09535562 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:00.874851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:00.875851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.876382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.876865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.888171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:00.890361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-8a4447cc-07fd-4125-b04c-fa48501f68ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:00.908133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:00.909700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:00.911124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:00.913827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 349/693] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:00.915306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41d90-c10d-4daf-83b0-6c64aad0de89 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:00.915670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41d90-c10d-4daf-83b0-6c64aad0de89 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:00.916107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb41d90-c10d-4daf-83b0-6c64aad0de89 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:00.917791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 350/694] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:00.925405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb41d90-c10d-4daf-83b0-6c64aad0de89 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:00.993781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:01.001676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b718e-5e7b-4147-bfaf-4fa6cd93f578 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:01.077602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:01.079103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.080135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.081110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.083716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:01.084525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:01.087623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b5cfe-64bf-445e-a92e-31826bb4eeab None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:01.095607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:01.102985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb41d90-c10d-4daf-83b0-6c64aad0de89 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:01.115669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 345/695] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:01.117000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.117729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.118422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.160449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:01.161098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:01.161537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.162077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.162579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.163548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:01.163992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:01.164891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:01.165645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:01.188836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 351/696] 10.222.0.65 () {56 vars in 1135 bytes} [Mon Aug 29 21:05:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:01.189928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:01.301647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 346/697] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:01.303587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:01.345097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0d13-4e4f-443e-9fba-282db01d3c66 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:01.409674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:01.411533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.412317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.413425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.427715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:01.430866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:01.435739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:01.451940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260890c-8a80-4dc1-9e76-edbdb6acf9a1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:01.458107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:01.459772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:01.462209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:01.466023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 352/698] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:01.480199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:01.518310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:01.534313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfa116-3308-40ea-acdf-550a6b9d50d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:01.534970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfa116-3308-40ea-acdf-550a6b9d50d3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.535259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfa116-3308-40ea-acdf-550a6b9d50d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.535542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfa116-3308-40ea-acdf-550a6b9d50d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.653670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d630b-c6b1-44de-a2d7-d1b7b9ea76b9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:01.657931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfa116-3308-40ea-acdf-550a6b9d50d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:01.659506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfa116-3308-40ea-acdf-550a6b9d50d3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:01.698612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:01.699243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.699546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.699972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.700966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:01.701269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:01.702854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943588b-5754-4bf8-a103-92e5dc95de35 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:01.725309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 353/699] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:01.727575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a5daf-a85e-4df5-811e-893bfa030dac None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.727926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a5daf-a85e-4df5-811e-893bfa030dac None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.728450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a5daf-a85e-4df5-811e-893bfa030dac None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.734674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 354/700] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:01.735699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a5daf-a85e-4df5-811e-893bfa030dac None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:01.780948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:01.798349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 347/701] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 21:05:01.798349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.798349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.798349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.825143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:01.825143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:01.848503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:01.888351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a5daf-a85e-4df5-811e-893bfa030dac None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:01.911425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 348/702] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:01.913327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:01.943783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:01.944853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:01.945488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:01.946049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:01.947778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3e5014eab4c7479f9ee493e31c91b3f4) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:01.948329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:01.950617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:01.968490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 355/703] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:01 2022] GET /identity/v3/role_assignments?user.id=3e5014eab4c7479f9ee493e31c91b3f4&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:01.968490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.121173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066ca3c-6276-4cb6-9602-f252a0d1d6bb None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.152540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:02.170662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.171467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.171759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.172068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.172673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.174089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:02.187352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:02.189414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:02.190293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 349/704] 10.222.0.65 () {56 vars in 1129 bytes} [Mon Aug 29 21:05:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:02.199272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.256777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c22331-ef4d-4050-b6e4-3d5d4b4888bd None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.310256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.310925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.311201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.311468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.322746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:02.335371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.345934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:02.367765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:02.369632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:02.371472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:02.371472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:02.371472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:02.380190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745d6a6-f7e0-427e-8b01-cdf3a7047849 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.380545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 356/705] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:02.382394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.427909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.428884 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.429662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.430127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.431208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:02.431922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.435297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.445271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 350/706] 10.222.0.65 () {56 vars in 1124 bytes} [Mon Aug 29 21:05:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:02.446922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.548336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:02.568060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b2d13-520f-4fdb-8ff3-8ea159b622f8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.634045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.635689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.636532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.637377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.659061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071541a6-82df-4728-95d7-db10dc0b8669 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.660482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.663763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:02.693405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:02.695253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:02.697474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:02.697474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:02.697474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:02.708190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 357/707] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:02.710095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.726269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.727312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.727779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.728275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.729387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:02.730148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.733356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.766756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 351/708] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:02.766756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.808832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:02.914001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14065ef1-6b91-4b12-bf6b-a2b914e861e5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.918615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2da95a-020e-41a8-bc25-623b864cb2b7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:02.929794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:02.960959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.961736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.962018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.962290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.962952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:02.963261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.964739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.978660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:02.979659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:02.985309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 352/709] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:02.985309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:02.985563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:02.985563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:02.985563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:02.985756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:02.987080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:02.995649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 358/710] 10.222.0.65 () {56 vars in 1124 bytes} [Mon Aug 29 21:05:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:02.996717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:03.140104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143f805-9435-4b5e-a3d6-b7af5ba1e1cb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:03.150002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:03.183868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80621-f414-4068-8054-f8cb8608fa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:03.184503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80621-f414-4068-8054-f8cb8608fa22 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:03.184838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80621-f414-4068-8054-f8cb8608fa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:03.185048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80621-f414-4068-8054-f8cb8608fa22 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:03.185671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80621-f414-4068-8054-f8cb8608fa22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:03.187622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80621-f414-4068-8054-f8cb8608fa22 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:03.219257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9500e1-9e34-431d-8199-36398e192624 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:03.298312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:03.300069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:03.301440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:03.302347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:03.318797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:03.322600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:03.339325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:03.340858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:03.342398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:03.345307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 353/711] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:03.353116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:03.369779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 359/712] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:05:02 2022] POST /identity/v3/users => generated 254 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:03.398278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:03.466479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c5c79-84ea-43fd-8b5d-a60de6d316e6 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:03.467289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c5c79-84ea-43fd-8b5d-a60de6d316e6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:03.468055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c5c79-84ea-43fd-8b5d-a60de6d316e6 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:03.476189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 360/713] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:03.478592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:03.479377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:03.480373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:03.521314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcd194-8b78-4840-8f6f-c411b55ebc96 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:03.525454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:03.527009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:03.569178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:03.569827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:03.570213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:03.570669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:03.571528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:03.571998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:03.573710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:03.590335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 354/714] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:03.598154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:03.724888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 361/715] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:03.750731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:03.766740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34872c65-a518-49b4-afc1-534348f1c98c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:03.808336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:03.816445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:03.816445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:03.816445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:03.816445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=fb437e7c06504d0aa7891269ed02f9e6) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:03.816445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:03.816445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:03.823133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 355/716] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:03 2022] GET /identity/v3/role_assignments?user.id=fb437e7c06504d0aa7891269ed02f9e6&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:04.020148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:04.166453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:04.175895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657fad4-6564-40e0-bd06-d62be63844a9 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:04.217812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:04.218473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:04.218754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:04.219041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] PATH_INFO: `/v3/services/network` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:04.219652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:04.221107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:04.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:04.236195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:04.237052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 362/717] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:04 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:04.243991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:04.412491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c25e3-7780-49f2-a5a7-9b1472863a07 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:04.456141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:04.457604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:04.458347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:04.459121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:04.460286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] RBAC: Adding query filter params (name=network) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:04.460898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:04.462888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2e1ce-5f16-4dd5-87a5-002f02825a1a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:04.471268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 356/718] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:04.481174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:04.621305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac8bc9-8389-420e-afd8-f94ccc32c165 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:04.622404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac8bc9-8389-420e-afd8-f94ccc32c165 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:04.622901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac8bc9-8389-420e-afd8-f94ccc32c165 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:04.629784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77c32-d557-43e3-89bc-0191d01e8779 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:04.630394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 357/719] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:04.635482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:04.636144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:04.636630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:04.685053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:04.685272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:04.685393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:04.685470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:04.685470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:04.685470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:04.685888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:04.686081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:04.687529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:04.700410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:04.705684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 363/720] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:04.800067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 358/721] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:05.246321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:05.273675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:05.395952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02149c6e-03bf-4736-ac7d-df620a7101c0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:05.445181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:05.445923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:05.446205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:05.446532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:05.462368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:05.467839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:05.469562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:05.471226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 364/722] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:05.478706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:05.600024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79718513-5dfa-4cc4-a554-f5ae7e2b427f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:05.640152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:05.640849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:05.641085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:05.641334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:05.642048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:05.642429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:05.644261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34ad8f-b04a-4713-b30f-570641615cbf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:05.653955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 359/723] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:05.661419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:05.708586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2983c0-7a56-49c3-8e0f-dc49b0232d7c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:05.708951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2983c0-7a56-49c3-8e0f-dc49b0232d7c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:05.709153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2983c0-7a56-49c3-8e0f-dc49b0232d7c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:05.710838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 360/724] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:05.716379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:05.720020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:05.720020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:05.738732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:05.740368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:05.744269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:05.826580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 361/725] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 21:05:05.839523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:05.866691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617e20b-6132-442d-886e-a69fb3892fa8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:05.920259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:05.921338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:05.921865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:05.922308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:05.940789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:05.942289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:05.965683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:05.966745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:05.967703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:05.967703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:05.967703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:05.974268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 365/726] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:05.985167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:06.134885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:06.166691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d6d4-b847-467f-aef0-53924fb421d7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:06.265671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:06.266510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.269714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.270174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.271247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:06.271745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:06.274346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:06.292330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:06.309153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 362/727] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:06.316722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:06.342846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ac341-e58a-4ac4-92fe-cf0118d6a916 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:06.410162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:06.415922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdaae8-af10-4513-9fbe-77e7458ca119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:06.416529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdaae8-af10-4513-9fbe-77e7458ca119 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.416812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdaae8-af10-4513-9fbe-77e7458ca119 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.417142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdaae8-af10-4513-9fbe-77e7458ca119 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.417987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdaae8-af10-4513-9fbe-77e7458ca119 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:06.420956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdaae8-af10-4513-9fbe-77e7458ca119 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:06.514624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854ea1b-1fbb-4005-aa69-7485684fe5cf None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:06.558631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:06.559307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.559593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.559924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.569212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:06.570746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-e5d13e1b-2d8e-4e50-b4e5-9ae6fe09d440 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:06.585394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:06.592831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 363/728] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:05:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5435b-4823-44c9-858e-60e3188fc499 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.594449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5435b-4823-44c9-858e-60e3188fc499 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.595460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5435b-4823-44c9-858e-60e3188fc499 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.595460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 364/729] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:06.595460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5435b-4823-44c9-858e-60e3188fc499 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:06.597897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 366/730] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:06 2022] POST /identity/v3/services => generated 251 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:05:06.602292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.603215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.603996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.633682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:06.635527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:06.737292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:06.780222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 367/731] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:06.829874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5435b-4823-44c9-858e-60e3188fc499 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:06.883724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:06.884837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.885354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.886012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.887031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:06.887925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:06.891032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bfbe2-23e9-4fec-9ad7-615c6e938bad None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:06.918824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 365/732] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:05:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:06.924781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:06.985950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbea0-4245-412f-8c62-2ca262eeca3f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.987123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbea0-4245-412f-8c62-2ca262eeca3f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.988156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbea0-4245-412f-8c62-2ca262eeca3f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:06.991108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 366/733] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:06.998711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:06.999026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:06.999271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.024911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:07.026133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:07.054135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737142-0ad2-4e41-b826-921fb5ded500 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.077030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:07.100978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.102086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.102628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.103127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.115269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 367/734] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:07.117348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:07.118839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:07.146301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:07.147609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:07.148651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:07.148651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:07.148651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:07.148651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 368/735] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:07.153767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.164546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.240285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:07.336804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11acb39-168e-4fa0-993a-52be2d4b01b6 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.379239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df784960-cf2a-4c53-9562-f64ba3648f44 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.413601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.414705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.415336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.416427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.417317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:07.417856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:07.419508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:07.435692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 368/736] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:07.441952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.458869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.459851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.460385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.460630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.474357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:07.481206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:07.482321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 369/737] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:07.483170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.523765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:07.576176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4bc43-4ba6-4342-907b-d2fe6ad1a291 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.584204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:07.613268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537024b-d764-405c-8375-c527c82a9d68 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.615682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.616351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.616691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.616999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.624771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:07.626973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-88f882b5-4294-4f51-81ad-441b4b652bd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:07.639869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:07.640819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:07.641723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:07.643788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 369/738] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:07.645794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa329a-413a-4665-a8d4-e3fcc212e005 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.646004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa329a-413a-4665-a8d4-e3fcc212e005 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.646280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa329a-413a-4665-a8d4-e3fcc212e005 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.648606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 370/739] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:07.652394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa329a-413a-4665-a8d4-e3fcc212e005 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.662246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095877f3-5907-436c-a357-4accd09506fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.662903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095877f3-5907-436c-a357-4accd09506fe None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.663187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095877f3-5907-436c-a357-4accd09506fe None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.663465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095877f3-5907-436c-a357-4accd09506fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.676685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095877f3-5907-436c-a357-4accd09506fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-095877f3-5907-436c-a357-4accd09506fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:07.682807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:07.683975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:07.685063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:07.692918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 370/740] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:07.694141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095877f3-5907-436c-a357-4accd09506fe None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.799273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:07.810489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095877f3-5907-436c-a357-4accd09506fe None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.852673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.853357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.853585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.853846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.854853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:07.855150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:07.856742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d64100-a4e2-4a80-b565-3812f3f022c5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:07.868861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa329a-413a-4665-a8d4-e3fcc212e005 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:07.870849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 371/741] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:07.872099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.872407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.872679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.893152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:07.895047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:07.917197 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:07.918008 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:07.918469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:07.919026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:07.919717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:07.920059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:07.921453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:07.930976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 371/742] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:07.931820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:07.967573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:07.992289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 372/743] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 21:05:07.993575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.055757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f43d-0643-471f-80a9-e82298502d67 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.103908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.104793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.105206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.105521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.106344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:08.106654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.108165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.114377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:08.120284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 372/744] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:08.121226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.158148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca584914-0c16-471a-871c-84820a1479e4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.197065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.198241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.198999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.199716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3e5014eab4c7479f9ee493e31c91b3f4/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.202679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=3e5014eab4c7479f9ee493e31c91b3f4, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.204450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.235679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:08.247013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b672c-8e0e-44e9-bd35-5e038559892b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.288166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.288822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.289190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.289471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.301818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.303335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.316217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 373/745] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:05:07 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3e5014eab4c7479f9ee493e31c91b3f4/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:08.320921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:08.321981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:08.322889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:08.322889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:08.322889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:08.322889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 373/746] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:08.323590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.324248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.466995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da83307-08f7-41c5-a5f0-eadfa3118794 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.482077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872e422-d2a6-4055-8c43-0f4140882523 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.523918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.523918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.523918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.523918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.524445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.527910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.535327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.536063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.536347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.536616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.548681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:08.551719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.553618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:08.571116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:08.572244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:08.573134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:08.573134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:08.573134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:08.573134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 374/747] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:08.573926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.642320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 374/748] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:08 2022] POST /identity/v3/services => generated 240 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:08.643335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.655175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:08.719340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed5504-f8e2-419b-b400-68d416c0cb03 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.776829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.777887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.778552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.779037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.779938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:08.780484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.785272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.801582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 375/749] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:08.807558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:08.898189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c516d-0a4e-4f79-996b-8f899ee8f16d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.898740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:08.956670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:08.957663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:08.958199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:08.958703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:08.959695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:08.960264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:08.961134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575e352-3296-41d1-ad33-b747407d8ca8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:08.962140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:08.982339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 375/750] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:08.988518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.021597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.022503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.023142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.023682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.036743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.039577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:09.060327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:09.061724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:09.063092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 376/751] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:05:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:09.069935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.074563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.132349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a2b3-4aae-417b-bfba-6ab5322f8a58 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.172441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.173432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.173899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.174366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.182838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.184760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:09.194316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a529e-875b-4ee8-943b-40433331d127 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-42ad53b8-4493-4334-8977-334e55da396f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:09.198375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:09.199299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:09.200436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:09.202673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 376/752] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:09.209228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36414f86-3783-4141-91fa-552f57863c01 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.209706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36414f86-3783-4141-91fa-552f57863c01 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.210161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36414f86-3783-4141-91fa-552f57863c01 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.211119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.213392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 377/753] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:09.214332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36414f86-3783-4141-91fa-552f57863c01 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.244582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.245684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.246172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.246730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.247788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:09.248137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.250027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1abd3-2c47-4fd0-be47-d5dfe75d34e9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:09.273073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 377/754] 10.222.0.65 () {56 vars in 1149 bytes} [Mon Aug 29 21:05:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:09.274779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.275158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.275670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.296107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:09.297409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:09.345659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36414f86-3783-4141-91fa-552f57863c01 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.373296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.380206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 378/755] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:09.381891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.388931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.389779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.390215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.390642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.391774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:09.392361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.394995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:09.416467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 378/756] 10.222.0.65 () {56 vars in 1135 bytes} [Mon Aug 29 21:05:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:09.422579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.503214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.509582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae22a5-180a-48ca-9e9b-a1acaf8c7946 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.542776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b11ec7-d7da-4589-bad5-81eda73c7412 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.549010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.549637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.550485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.550759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.585313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.586377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.586842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.587299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.634562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.636271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:09.660318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.667682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.669466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:09.791844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 379/757] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:05:09.798470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.864343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.877514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 379/758] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:05:09.878678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:09.924553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:09.941316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d6877-bfb9-4652-91a8-8ce93405d80b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.980623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:09.981603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:09.981994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:09.982364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:09.990965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614e7ac-e0a2-4978-be39-76c2dbec8fd4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:09.997504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:09.999949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:10.018037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:10.019609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:10.020548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:10.020548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:10.020548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:10.024229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 380/759] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:10.025380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.039050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.039693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.040402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.040685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.041285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.043370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:10.058177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:10.059270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:10.060208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:10.064205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 380/760] 10.222.0.65 () {56 vars in 1129 bytes} [Mon Aug 29 21:05:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:10.066757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.156310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:10.187338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db399b-1c88-42b2-8b20-3a349726d0f4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.219625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ebfa9-4613-471d-ae91-570fb70e454c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.226536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:10.231711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.232488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.232835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.233183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.247249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.248912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:10.268334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:10.270092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:10.272076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:10.272076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:10.272076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:10.275946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 381/761] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:10.275946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.291654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.292873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.293302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.293774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.294727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:10.295297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.297709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.315016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 381/762] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:10.316687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.411854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232115-f92e-4628-ad48-86818ce673b3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.446690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d955ed0-57e4-4bfb-a017-2c5e61d725bb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.460512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.461800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.462248 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.462732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.463787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:10.464287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.466467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.471343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:10.473779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 382/763] 10.222.0.65 () {56 vars in 1124 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:10.474900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.492269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.493164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.493577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.493964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.494767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:10.495177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.496785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.517486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 382/764] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:10.518864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.655121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fce20-57ff-4c1f-8fd7-300e3bcf0214 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.675466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:10.701066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.701940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.702385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.702798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.703573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:10.704069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.706157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.713077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 383/765] 10.222.0.65 () {56 vars in 1124 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:10.714314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.737747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f781d84-da4d-474e-ac0a-931747c7078e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.772077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:10.774413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.775055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.775336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.775611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.783082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.785189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:10.797141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:10.798126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:10.799121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 383/766] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:10.800274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.842353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4066e-a7aa-4115-9b0b-78b3d1c6385e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.884224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.885724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.886206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.886715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.895396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.896890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e0037d60-47b4-4088-8a31-027eacfc6149 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:10.909795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:10.910730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:10.911653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 384/767] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:10.913065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade99fd-b8ef-431a-85ba-87c54b93f55d None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.913065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade99fd-b8ef-431a-85ba-87c54b93f55d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.913065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade99fd-b8ef-431a-85ba-87c54b93f55d None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.914142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 385/768] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:10.918760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade99fd-b8ef-431a-85ba-87c54b93f55d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:10.922699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f856f-b524-4813-8d2c-1a42e11b5b44 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:10.934190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:10.962903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:10.963548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:10.963832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:10.964155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:10.964871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] RBAC: Adding query filter params (interface=public, service_id=ea247be52ce44bc9b9f32c743fda18ab, region_id=RegionOne) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:10.965176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:10.966598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:10.976770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 384/769] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/endpoints?service_id=ea247be52ce44bc9b9f32c743fda18ab&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:10.977692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.035771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade99fd-b8ef-431a-85ba-87c54b93f55d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.038125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:11.071249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:11.072052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.072272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.072946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.073909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:11.074460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:11.076519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97312a-b9a9-499f-84a0-a9c6a9050a29 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:11.085533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02e951-fdc2-42a0-8d8b-41701d010057 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.090460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 386/770] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:11.091791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.092120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.092387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.112542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:11.113726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:11.123287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:11.123924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.124213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.124494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.127766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:11.127766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:11.127766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf458e-1926-4684-8bf3-35c042ad9750 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:11.139867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 385/771] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:11.141141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8010c-df8f-4cec-a5b3-6f0e7180ec55 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.141432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8010c-df8f-4cec-a5b3-6f0e7180ec55 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.141707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8010c-df8f-4cec-a5b3-6f0e7180ec55 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.143185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 386/772] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:11.144421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8010c-df8f-4cec-a5b3-6f0e7180ec55 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.181241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:11.189807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 387/773] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:11.190686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.263618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8010c-df8f-4cec-a5b3-6f0e7180ec55 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.294010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:11.302571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83624-998a-4d99-b1ed-a625a77bd9c8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.302927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:11.303550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.303831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.304139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.304751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:11.306123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:11.311191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 387/774] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:11 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:11.312005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.359363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:11.360371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.360850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.361573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.363584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=fb437e7c06504d0aa7891269ed02f9e6) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:11.364169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:11.366083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095bac4-1d36-4aea-acb3-24a766db50fb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:11.374935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 388/775] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:11 2022] GET /identity/v3/role_assignments?user.id=fb437e7c06504d0aa7891269ed02f9e6&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:11.377464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.378975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.379490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.402240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:11.404067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:11.438620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:11.446074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84d2ad-34d6-4cd2-941b-de6f320c9360 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.484155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 389/776] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:11.484574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:11.485268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.485552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.486395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.486586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3c2842cc986347dc802236ee27b9de50) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:11.486891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:11.489469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:11.499298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 388/777] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:11 2022] GET /identity/v3/role_assignments?user.id=3c2842cc986347dc802236ee27b9de50&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:11.538440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.652296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:11.667456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368bc81-eb32-4542-acbb-a000660e106e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.710511 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:11.711151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:11.711441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:11.711714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:11.723282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:11.728220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:11.730032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:11.730883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 390/778] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:11.735494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.799190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:11.941588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7afed-1a4f-4716-89b6-1e59fc996286 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:11.947844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166b183-86c6-4e9e-b9d7-6bbfa45e5b30 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:12.018773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.019324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.019603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.020039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.020837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.021309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:12.021634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.021746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.021926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.022231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.022904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.023939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.024479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.034963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 389/779] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:12.043094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:12.044209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:12.045153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 391/780] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:11 2022] GET /identity/v3/services/network => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:12.046017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:12.064993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:12.080589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:12.106811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:12.200864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5fac1-5062-4757-b537-005f0eee8508 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:12.240422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.242239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.242531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.242799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.263399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.264961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:12.284848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:12.285860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:12.286865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:12.286865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:12.286865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:12.286865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 390/781] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:12.294108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:12.321956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ecd5-50a1-46a8-a684-abfe945d09e7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:12.362160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.362800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.363083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.363363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.364029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] RBAC: Adding query filter params (name=network) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:12.364338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.365761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.372117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 392/782] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:12.378837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:12.499987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:12.512437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d2c1f-5da7-461d-9e64-5f2b81767c58 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:12.568623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:12.597938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.598853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.599276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.599700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.601011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] RBAC: Adding query filter params (type=network) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:12.601553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.604464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.615850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 393/783] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:12.622680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:12.652318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed58b4-eee0-4366-a92c-e1afa93d088e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:12.749260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.750300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.750791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.751271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.752449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:12.752973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.759619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.794425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 391/784] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:12.797337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:12.862913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7fc45-7d95-4843-b23a-de9c1fc84f9e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:12.961087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:12.962269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:12.962630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:12.963116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:12.970856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] RBAC: Adding query filter params (interface=public, service_id=fb8b17dfd647465a94bddfcef31e8e51, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:12.970856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:12.970856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:12.991368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 394/785] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/endpoints?service_id=fb8b17dfd647465a94bddfcef31e8e51&interface=public®ion_id=RegionOne => generated 194 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:12.997971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:13.024484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:13.025904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24e348-4918-4e37-a50d-1bdf9a7d665b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:13.116547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:13.117110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:13.117394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:13.117669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:13.120340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:13.139045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:13.140859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:13.202086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:13.203966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:13.206385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:13.209319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5e896-f1a9-4a3b-8543-7fe85882fd42 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:13.218145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 392/786] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:13.220323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:13.296849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:13.297701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:13.298028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:13.298270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:13.298867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:13.301225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:13.309459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 395/787] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:12 2022] GET /identity/v3/services => generated 772 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:13.421183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622011ca-c440-4185-994a-a960075c97f3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:13.477841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:13.478493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:13.483960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:13.483960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:13.483960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:13.483960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:13.483960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:13.525015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 393/788] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:05:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:13.530668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:13.650818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:13.722910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6184e9-25ec-4b93-9345-a493910f7d9d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:13.796084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881299f-a723-4309-8431-33f10efe68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:13.797107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881299f-a723-4309-8431-33f10efe68d2 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:13.797585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881299f-a723-4309-8431-33f10efe68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:13.798067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881299f-a723-4309-8431-33f10efe68d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:13.942498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881299f-a723-4309-8431-33f10efe68d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:13.944987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881299f-a723-4309-8431-33f10efe68d2 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:14.082368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:14.159654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 396/789] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:05:14.165304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:14.201921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a6ddd-7da4-4bbd-91be-142ed63e0bba None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.202382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a6ddd-7da4-4bbd-91be-142ed63e0bba None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.202851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a6ddd-7da4-4bbd-91be-142ed63e0bba None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.206375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 397/790] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:14.215841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.216346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.216816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.250987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:14.252221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:14.274477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:14.368252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0ac6-1047-4ab5-96c2-f89c24313992 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:14.394939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 398/791] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:14.428338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:14.428982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.429254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.429521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.445292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:14.446831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-8d6dcba3-0605-4fa6-aaf8-135b2d27f7eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:14.463002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:14.465628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:14.467621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:14.467621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:14.467621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:14.467621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 394/792] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:14.472722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:14.604215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:14.609452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee68d9c-00b7-4bfd-a124-b7e32ef4f9f6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:14.661038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa43d1e-1279-4d3d-9f8e-389aef3d6899 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.661428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa43d1e-1279-4d3d-9f8e-389aef3d6899 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.661677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa43d1e-1279-4d3d-9f8e-389aef3d6899 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.662820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 395/793] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:14.665543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:14.666170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.666449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.666726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.667379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:14.667680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:14.669120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdc4d0-4a2c-48aa-b512-9a63c9101ee7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:14.675687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e33b9-7b27-461e-b58b-17b69fdcd172 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.676083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e33b9-7b27-461e-b58b-17b69fdcd172 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.676314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e33b9-7b27-461e-b58b-17b69fdcd172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.685449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 399/794] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:14.687386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367971c7-b1b8-4eda-97a0-026d413f8dc3 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.687386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367971c7-b1b8-4eda-97a0-026d413f8dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.687386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367971c7-b1b8-4eda-97a0-026d413f8dc3 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.689043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 400/795] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:14.699025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367971c7-b1b8-4eda-97a0-026d413f8dc3 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:14.704896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-632e33b9-7b27-461e-b58b-17b69fdcd172 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:14.706156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-632e33b9-7b27-461e-b58b-17b69fdcd172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:14.816405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 396/796] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:14.818130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.818617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.819111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.838447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367971c7-b1b8-4eda-97a0-026d413f8dc3 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:14.846834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:14.854871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:14.892584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:14.954025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:14.955564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:14.956065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:14.956504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:14.968547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:14.971407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:14.990005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 397/797] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:14.991578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:14.993116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:14.993998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:14.994891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:14.995469 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 401/798] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:15.009842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.054912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:15.131793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dde7b7-531a-4187-aa3b-9384719dffc1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.174978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99d65b-436c-48bb-b0cb-ec59fd70db84 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.180766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.181538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.181921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.182297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.182990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:15.184537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:15.196487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:15.306254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:15.307106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 398/799] 10.222.0.65 () {56 vars in 1129 bytes} [Mon Aug 29 21:05:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:15.307938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.308011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.308011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.308011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.308011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.308317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:15.308317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:15.308317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:15.308503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:15.308571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 402/800] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:15.308571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.326491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc950-826d-46be-b374-71f1f8cc59e7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.350218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:15.367328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.368033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.368313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.368594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.376478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4043f3-fc33-4840-ad7f-78db083044a8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.383022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:15.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:15.403231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 399/801] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:15.404394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.426187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.426933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.427204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.427538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.428146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:15.428446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:15.429844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:15.436147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 403/802] 10.222.0.65 () {56 vars in 1124 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:15.437315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.516001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:15.529843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa4042-1ed8-4485-acb6-181a3b6278e3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.577301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.577926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.578208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.578483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.579218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3e5014eab4c7479f9ee493e31c91b3f4) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:15.579527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:15.581223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:15.599922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 400/803] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/role_assignments?user.id=3e5014eab4c7479f9ee493e31c91b3f4&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:15.599922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.600312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dcfbc-7639-4523-be4e-cfaafcac3b37 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.650133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.650895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.651255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.651606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.670604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:15.679630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:15.680897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:15.682204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 404/804] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:15.682950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.692281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:15.753586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d72dbd-4804-49c6-ac21-ff4a4a613661 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.802715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.803339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.803614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.804239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.804603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:15.804907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:15.806358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:15.821848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 401/805] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:15.822850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.842024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:15.858607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87dbe8-5f3e-4c92-b59b-6864a5ee5157 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:15.924962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:15.926310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:15.927041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:15.927742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:15.929018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:15.929766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:15.932253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:15.941373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 405/806] 10.222.0.65 () {56 vars in 1124 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:15.943027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:15.979393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c39e4b-7028-49be-b1fd-b9c132a7ffd8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.026771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.027492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.027778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.028080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.028784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:16.029092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.030544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.042881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 402/807] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:16.044238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:16.155401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:16.170667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:16.198171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dacfc8-387c-44ff-a798-68ad81b3748c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.219366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f95e1-5749-498c-971e-4f6e50a4efaa None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.241548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89375f18-4d66-498a-85de-b2dbfdfe8489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.242573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89375f18-4d66-498a-85de-b2dbfdfe8489 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.243059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89375f18-4d66-498a-85de-b2dbfdfe8489 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.243537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89375f18-4d66-498a-85de-b2dbfdfe8489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.244571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89375f18-4d66-498a-85de-b2dbfdfe8489 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.246252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89375f18-4d66-498a-85de-b2dbfdfe8489 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.266154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.269800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.269800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.269800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.283412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.284896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:16.301623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:16.305864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:16.308674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:16.308674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:16.308674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:16.308674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 406/808] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:16.308674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:16.383079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 403/809] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:16.384436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfd25-a719-4619-b633-f9af9def0749 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.384731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfd25-a719-4619-b633-f9af9def0749 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.385012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acfd25-a719-4619-b633-f9af9def0749 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.390940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 404/810] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:16.390940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acfd25-a719-4619-b633-f9af9def0749 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:16.434860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba9785-d9cf-4136-a7d2-1be315ea9455 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.469945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:16.503964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.505205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.505653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.506802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.522032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.524311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-20a4e406-09f4-4e15-8bcf-b2febe6f2dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:16.542364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:16.543525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:16.544804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:16.544804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:16.544804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:16.546628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 407/811] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:16.548335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.549391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.549854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.572746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:16.574326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:16.591804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acfd25-a719-4619-b633-f9af9def0749 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.638289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:16.641742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.641939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.642163 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.642421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.643121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:16.643428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.644974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.660580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 405/812] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:16.661536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:16.672394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 408/813] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:16.673390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:16.782623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:16.790567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46105b-67f3-4578-bc3b-56bf589f191f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.832681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.833628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.834062 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.834483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.843814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.845749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:16.861944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:16.863509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:16.864534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:16.867908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 409/814] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:05:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:16.882919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:16.903102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488173ea-0d96-4419-98f9-91dd5024b367 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:16.951259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:16.952073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:16.952354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:16.952628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:16.953279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:16.953581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:16.954962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:16.982952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 406/815] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:16.988815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.062994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.078482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9b5b2-b521-4437-9cef-86a7804ce870 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.116833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d623316-059a-4318-bdfc-1ec7dd09a3a3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.119653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.120329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.120677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.120968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.121742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:17.122053 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.123729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.152247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 410/816] 10.222.0.65 () {56 vars in 1149 bytes} [Mon Aug 29 21:05:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:17.153447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.161995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.162650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.162930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.163212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.175551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.176151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.178235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:17.191126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:17.192204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:17.193164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:17.195715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 407/817] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:17.196851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.270576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c539-7a06-486c-bd2c-8979a2f6a6fd None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.321655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.322399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.322627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6ae37-c89e-4032-9868-5efdf7297deb None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.323233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.323233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] PATH_INFO: `/v3/services/network` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.323734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.325673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:17.339102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:17.340306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 21:05:17.341139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:17.344077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 411/818] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:17.345367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.346464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.372429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.373030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.373314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.373589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.388616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.390364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:17.412416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:17.413485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:17.414394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:17.414394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:17.414394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:17.418560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 408/819] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:17.419665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.461081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.493125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138d400-3e93-4ea6-a32b-6bcf1316778a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.532969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.534595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.535194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.535673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.536684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:17.537215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.538780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e61f-5c71-42fe-a03e-0e798c38dfcf None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.539122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.560061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 412/820] 10.222.0.65 () {56 vars in 1135 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:17.561359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.577955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.578633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.579068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.579332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.580020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:17.580396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.582135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.592263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 409/821] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:17.592263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.634223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.707230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa746987-4d55-4851-859c-86cce6a75edb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.734091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65713446-3bf7-4b00-8c83-fba76d0383c9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.749558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.750650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.751199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.751778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.752913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:17.753422 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.755413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.758545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.771984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 413/822] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:17.773205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.776901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.777747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.778016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.778109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.792394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.793901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4372defa-133c-4a65-ad2b-d75c8c312cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:17.812274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:17.813333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:17.814212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:17.814212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:17.814212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:17.814212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 410/823] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:17.815160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1b0b-614b-4ed8-bf28-1ae3aca3c092 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.815160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1b0b-614b-4ed8-bf28-1ae3aca3c092 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.815339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf1b0b-614b-4ed8-bf28-1ae3aca3c092 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.816754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 411/824] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:17.817297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1b0b-614b-4ed8-bf28-1ae3aca3c092 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.917343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:17.921140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ba9c-9955-4998-b57e-d640d44d69bc None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.948115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf1b0b-614b-4ed8-bf28-1ae3aca3c092 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:17.961363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.961915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.962197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.962470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.963158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:17.963473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.965082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:17.970594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 414/825] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:17.976514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:17.988503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:17.989178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:17.989461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:17.989731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:17.997188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:17.998952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-2dec01ca-50ca-4fc9-9f85-6a3d697afaa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:18.014444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:18.015617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:18.016554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:18.018401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 412/826] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:18.019758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:18.020168 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:18.020513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:18.039745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:18.041275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:18.053270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:18.105700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7640c4d-b7b7-42fa-abc4-a4bd4eb526f5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:18.146458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:18.147070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:18.147350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:18.148253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 413/827] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:18.148359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:18.148979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:18.151262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:18.151262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:18.151262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:18.172543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 415/828] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:18.173432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:18.230301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:18.267031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02377829-6a48-463d-be2f-3c4a55435cc2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:18.289449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64648b-1e2c-409f-bb07-efce5dc13c4c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:18.309621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:18.310614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:18.311092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:18.311553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:18.312779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:18.313803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:18.316001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:18.327921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:18.328543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:18.328828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:18.329167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:18.330660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:18.330971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:18.332470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:18.349762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 416/829] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:18.349762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:18.456425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633637c-a59a-4611-9b2f-7db3783193a9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:18.490917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:18.491458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:18.491742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:18.492054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:18.498757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:18.500237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:18.513114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:18.514129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 417/830] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:18.521003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:18.557135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:18.615841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc4b10-b97c-4596-b5f7-60ed42784cdb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:18.648812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:18.649470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:18.649747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:18.650019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3c2842cc986347dc802236ee27b9de50/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:18.664547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=3c2842cc986347dc802236ee27b9de50, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:18.666079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:18.811297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 414/831] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:18 2022] POST /identity/v3/endpoints => generated 337 bytes in 663 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:18.812127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:18.883980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 418/832] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:05:18 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3c2842cc986347dc802236ee27b9de50/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:05:18.883980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:18.974380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e3981-4d28-4011-b936-a67c5648de94 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.015929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:19.018695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:19.027095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.027835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.028117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.028373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.028990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.030439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.036594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459de74-73c8-48b8-bcd1-62f09f828661 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1b4e3246-d379-4ff6-a7b2-419e7c98591e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:19.044373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:19.045586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:19.046595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 415/833] 10.222.0.65 () {56 vars in 1132 bytes} [Mon Aug 29 21:05:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:19.047564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27f050-b754-4560-9afa-dea2ef7504fe None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.047564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27f050-b754-4560-9afa-dea2ef7504fe None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.047564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27f050-b754-4560-9afa-dea2ef7504fe None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.053068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 416/834] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:19.056019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27f050-b754-4560-9afa-dea2ef7504fe None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:19.099082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.099844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.100180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.100505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.101326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:19.101702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.103281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22369292-6e90-4d04-ba88-21a0a7cb3bb3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.123356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 419/835] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:19.125204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.125515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.125783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.148052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:19.149254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:19.192024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27f050-b754-4560-9afa-dea2ef7504fe None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.230221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 420/836] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:19.231165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:19.234363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.234986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.235269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.235540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.240076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:19.240076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.240076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.244759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 417/837] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:19.250328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:19.310511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:19.323526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:19.345152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a907d-a11f-4ca3-8af6-1352f02badc9 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.373452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352fd55-1c78-4664-b19a-568ba0ec4ad7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.386004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.386709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.387019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.387306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/fb437e7c06504d0aa7891269ed02f9e6/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.389539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=fb437e7c06504d0aa7891269ed02f9e6, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.391037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.416983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.418039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.418552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.419048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.420365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:19.420897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.422499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.430323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 418/838] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:19.507289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 421/839] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:05:19 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/fb437e7c06504d0aa7891269ed02f9e6/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:05:19.508431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:19.690283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423a401-77d2-4dd1-8ade-1d3b34b468ac None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.708652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:19.740931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.741935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.742395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.742836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.743689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.745349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:19.759143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:19.760374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:19.761297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:19.765512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 419/840] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:19.767001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:19.893965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f619f-f6ca-45c5-a62e-309a193b98d3 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:19.934168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:19.934793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:19.935066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:19.935333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:19.936123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:19.936440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:19.937897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67f395-98ee-4667-9e7d-17fc10ef6c66 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:19.944514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 422/841] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:19.949923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:20.093227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1568a46-3e7f-4c73-8c4d-9ba2b1c0bcb9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:20.138160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:20.138867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:20.139152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:20.139427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:20.140262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:20.140571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:20.142099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2814f-27b6-4628-baf9-9bd4cfc11e1b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:20.148566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 420/842] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:21.015453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893e64-5673-4bb4-85a5-17099a10a673 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.016042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893e64-5673-4bb4-85a5-17099a10a673 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.016042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893e64-5673-4bb4-85a5-17099a10a673 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.017664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 423/843] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:21.022988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.023468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.023575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.043029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:21.044276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:21.083343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:21.119103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 421/844] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:21.399612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893e64-5673-4bb4-85a5-17099a10a673 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:21.403738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:21.513792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893e64-5673-4bb4-85a5-17099a10a673 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:21.552294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fc71e-e3b5-41f0-857f-2adca8d0a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:21.553291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc71e-e3b5-41f0-857f-2adca8d0a18d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.553761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc71e-e3b5-41f0-857f-2adca8d0a18d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fc71e-e3b5-41f0-857f-2adca8d0a18d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.567247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fc71e-e3b5-41f0-857f-2adca8d0a18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-450fc71e-e3b5-41f0-857f-2adca8d0a18d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:21.573810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:21.574799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:21.576222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 424/845] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:05:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:21.581651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:21.609344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f4841-a5c1-42dc-9040-e2320fd4cda9 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.609633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f4841-a5c1-42dc-9040-e2320fd4cda9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.609913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f4841-a5c1-42dc-9040-e2320fd4cda9 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.612787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 425/846] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:21.618548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4a436-2720-466f-b72c-07f1b9888df6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.618834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4a436-2720-466f-b72c-07f1b9888df6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.619117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4a436-2720-466f-b72c-07f1b9888df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.640455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-bbe4a436-2720-466f-b72c-07f1b9888df6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:21.641410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bbe4a436-2720-466f-b72c-07f1b9888df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:21.713237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:21.713581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000d47a-cf52-467c-aa03-1e783ef3845b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:21.722550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 426/847] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:21.728805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0853e-07e0-4b41-b379-a9564a671bb3 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.728805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0853e-07e0-4b41-b379-a9564a671bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.728805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0853e-07e0-4b41-b379-a9564a671bb3 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.728805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 427/848] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:21.731555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.731839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.732178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.752834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:21.754077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:21.757970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:21.758809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.759241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.759692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.760550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:21.761002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:21.762536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:21.772827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 422/849] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:21.778544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:21.824379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:21.860520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 428/850] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:21.917665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d127fb9-4fe0-47e2-b265-33ad5790e386 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:21.955953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:21.957074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:21.957565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:21.958045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:21.969776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:21.972115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:21.989411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:21.990429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:21.991339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:21.991339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:21.991339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:21.993637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 423/851] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:21.999177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.029886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.123678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae64b34-9043-4715-b555-fe081ece7290 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.154282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:22.157828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:22.165633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.166253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.166532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.166807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.167452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:22.167756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.169194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.173346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a8b45-c5d0-46bb-9ff1-09d3b1b9ea70 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.180555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 429/852] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:22.181404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.209834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557731a-1acb-4a35-97e9-905d3744573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.210449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557731a-1acb-4a35-97e9-905d3744573b None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.210731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557731a-1acb-4a35-97e9-905d3744573b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.211014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557731a-1acb-4a35-97e9-905d3744573b None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.211674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557731a-1acb-4a35-97e9-905d3744573b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.213110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557731a-1acb-4a35-97e9-905d3744573b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.289565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ffbc5-5a02-4384-952e-aacd5821b8fa None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.290678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 424/853] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:22 2022] POST /identity/v3/services => generated 269 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:05:22.291967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711027fc-a8fe-435e-843a-b9b95e8c60d8 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.292260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711027fc-a8fe-435e-843a-b9b95e8c60d8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.292539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711027fc-a8fe-435e-843a-b9b95e8c60d8 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.293826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 425/854] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:22.294575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711027fc-a8fe-435e-843a-b9b95e8c60d8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.343866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430a9c8-3c83-4e81-b493-c4a103f6dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.344684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a9c8-3c83-4e81-b493-c4a103f6dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.345055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a9c8-3c83-4e81-b493-c4a103f6dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.345838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a9c8-3c83-4e81-b493-c4a103f6dbe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.360185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430a9c8-3c83-4e81-b493-c4a103f6dbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-8430a9c8-3c83-4e81-b493-c4a103f6dbe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:22.366878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:22.367995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:22.368992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:22.373541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 430/855] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:22.375207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.375207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.375441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.398000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:22.399740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:22.429105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711027fc-a8fe-435e-843a-b9b95e8c60d8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.444540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:22.470758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.471675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.471978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.472256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.482002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.483474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.489505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 431/856] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:22 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:22.490419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:22.499349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:22.500536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:22.501564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:22.504125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 426/857] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:05:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:22.509669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.527411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:22.613619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eeafc-3728-41c4-a8de-125f209147f4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.654657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.655394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.655753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.656129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.656879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:22.657258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.659425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.661948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1c3a5-058f-4bf6-b3e1-039a039d07ae None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.670438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 432/858] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:22.677176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.709081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.709705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.709988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.710264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.711051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:22.711360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.712869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.728322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:22.738742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 427/859] 10.222.0.65 () {56 vars in 1149 bytes} [Mon Aug 29 21:05:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:22.741315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.815776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:22.820248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa1cd1-56bc-4e50-95eb-be15f20a2d79 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.855148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679e83d-02f5-4392-a2c5-20027f947b6d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:22.858127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.859023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.859305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.859583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.871036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.872470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:22.889984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:22.891037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:22.892566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:22.892566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:22.892566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:22.892566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 433/860] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:22.892566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:22.896903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:22.897392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:22.897681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:22.897961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:22.979749 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:22.981263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:22.997696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:23.015299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df38fd5-5a10-4b5b-bb7f-365f087d1d3e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.058939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:23.062999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.063446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.063879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.064224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.064897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.066362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.106636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 428/861] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:22 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:05:23.106636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.211323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577da0d-6303-459f-b8e8-f4379d2f324e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.228714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 434/862] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:22 2022] POST /identity/v3/services => generated 251 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:23.230212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.272552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.273188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.273470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.273743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.274998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:23.275315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.277558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.297747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 429/863] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:23.304154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.337512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:23.393446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c7dd4-0252-4790-b8ad-74f08df59fb6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.450059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.451060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.451683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.452214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.464975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94de62a-ac83-40e9-a2e6-68b0127f63e6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.467209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.469450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.480325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:23.487763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:23.494745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:23.495976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:23.495976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:23.495976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:23.495976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 435/864] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:23.495976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.516027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.518406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.519059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.519556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.533716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.535976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:23.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:23.553115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 21:05:23.554079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:23.557536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 430/865] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:23.563632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.622240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8988-093c-4a70-b665-e00d2a71a85e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.668927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.669628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.669972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.670516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.671284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:23.671594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.673789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.684448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:23.694245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 436/866] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:23.699990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.716566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85085e-12a3-4ab1-8442-f27c164dc921 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.765018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:23.768734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.770047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.770618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.771039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.772116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:23.772960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.774965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.802265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 431/867] 10.222.0.65 () {56 vars in 1135 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:23.803461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.820658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96971673-60c4-46f3-8325-020890faaba4 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.869166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.869840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.870270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.870553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.879007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:23.880787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:23.896201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:23.897372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 437/868] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:23.903268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:23.929614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb43a3-bf37-4999-ae68-ff0dd094f30e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:23.968866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:23.970007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:23.970488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:23.970981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:23.980506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:24.045861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c7b14-f6ba-461b-b55b-e25604019efa None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:24.049921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.051081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.069682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:24.093236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20978c36-fd00-441e-8159-a51971269634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:24.093971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20978c36-fd00-441e-8159-a51971269634 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.094317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20978c36-fd00-441e-8159-a51971269634 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.094656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20978c36-fd00-441e-8159-a51971269634 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.097877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20978c36-fd00-441e-8159-a51971269634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:24.098387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20978c36-fd00-441e-8159-a51971269634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.099962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20978c36-fd00-441e-8159-a51971269634 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.115926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 438/869] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:24.122162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20978c36-fd00-441e-8159-a51971269634 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.191831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 432/870] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:23 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:05:24.198769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.256041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20978c36-fd00-441e-8159-a51971269634 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:24.260367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:24.307041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:24.307658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.307964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.308244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.309476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3c2842cc986347dc802236ee27b9de50) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:24.309812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.311385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e4e80-f3f6-426c-a5d1-ef49322f949a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.320225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 439/871] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/role_assignments?user.id=3c2842cc986347dc802236ee27b9de50&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:24.334784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be2cfc-cf8b-4177-8bac-051bf2bae3ed None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.335141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be2cfc-cf8b-4177-8bac-051bf2bae3ed None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.335312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be2cfc-cf8b-4177-8bac-051bf2bae3ed None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.338300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 440/872] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:24.342907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.343222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.343462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.347586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc27a43-6cf9-4ea0-9f42-3866ed6a3153 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:24.369345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:24.370784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:24.390624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:24.392236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.392727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.393310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.405341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.408620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.420794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:24.426851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:24.427942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:24.434101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:24.434101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:24.434101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:24.434101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 433/873] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:24.436692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.470712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 441/874] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:24.555755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:24.567493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92db043-4312-48a9-8df2-28c038769e4d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:24.611467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:24.612126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.612409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.612683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.613336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:24.613636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.615035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.628995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 434/875] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:24.635049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.757160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.762300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a594bb-25bf-45f7-8551-0fdb363f0bd4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:24.809155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:24.810079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.810546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.810984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.819967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.821879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:24.836391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:24.837591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:24.838668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:24.841000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 442/876] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:24.847672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.888129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:24.907118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b417b-4686-40a0-8161-d417998a7839 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:24.960429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:24.961490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:24.961834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:24.962110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:24.962913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:24.965051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:24.977278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:24.978771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:24.979836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 435/877] 10.222.0.65 () {56 vars in 1132 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:24.987391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:24.994648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e3d6c-ae0e-4473-9f7c-f96bfd506208 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.008433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:25.046501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.047826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.049494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.050009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.051373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:25.051935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.054198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.071621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 443/878] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:25.078932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:25.137650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee88e5-ced5-4840-92ec-5d81cb4dc58c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.182045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.182269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.182645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.183023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.183864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:25.184419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.186641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.196064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 436/879] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:25.200527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:25.229888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:25.244068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d486fc4-4522-412c-9ffc-be731c0b7a6e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.292827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.294660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.295146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.295617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.296718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=fb437e7c06504d0aa7891269ed02f9e6) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:25.297425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.300462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0faf30-d680-4ff9-8ef5-de3ea387e0d1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.310032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 444/880] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:25 2022] GET /identity/v3/role_assignments?user.id=fb437e7c06504d0aa7891269ed02f9e6&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:25.349292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:25.374349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e10b-b612-4bab-b974-adeebf5b74bc None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.374621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e10b-b612-4bab-b974-adeebf5b74bc None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.374899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e10b-b612-4bab-b974-adeebf5b74bc None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.376448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 445/881] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:25.380359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe51d2-50f9-42d1-98d9-0b1e0f47f388 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.382807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.383301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.383646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.405765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:25.407139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:25.434770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.435468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.435757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.436072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.436733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:25.437042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.439587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.447449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 437/882] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:25.455380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:25.518071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 446/883] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:25.603868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae47ec6-d815-4c14-80f7-87df48d58b51 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.652224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.652863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.653140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.653417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.654448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] RBAC: Adding query filter params (interface=public, service_id=f99003e55c90494d99df809e4c9d0549, region_id=RegionOne) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:25.654824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.656434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.663681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 438/884] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:25 2022] GET /identity/v3/endpoints?service_id=f99003e55c90494d99df809e4c9d0549&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:25.669424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:25.774353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:25.793766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a2381-4d1a-4f82-a682-4ebf3eee0f86 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.794782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:25.806872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:25.846358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.847584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.848073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.848809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.849884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.851640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.860706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 447/885] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:25 2022] GET /identity/v3/services => generated 1268 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:05:25.934234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e89cd-87f4-49c6-818a-4e7911cd16de None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:25.977975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:25.979186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:25.979688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:25.980146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:25.980987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:25.982606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:25.993200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:25.994508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:25.995551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:25.998172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 439/886] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:26.005084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:26.151949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65a369-f18e-49f9-8c5c-d801400fef3a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:26.192309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:26.193600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:26.194079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:26.194546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:26.195435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:26.195969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:26.197638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:26.203745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 448/887] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:26.211051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:26.296141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:26.336256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecea1b9-a74d-4d36-9f6f-5c1732e1aac8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:26.380330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:26.381014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:26.381244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:26.381522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:26.382165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:26.382474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:26.383864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:26.391084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 440/888] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:26.396841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:26.414614 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:26.512552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a872fc-324c-4cdf-9d50-81201003fc99 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:26.552481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:26.553504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:26.554011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:26.554484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:26.555406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] RBAC: Adding query filter params (interface=public, service_id=600b0c07b2d7435f934843a17b7effc7, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:26.555957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:26.557581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f7fa-55c9-42af-a203-c8c637189367 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:26.564885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 449/889] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:26 2022] GET /identity/v3/endpoints?service_id=600b0c07b2d7435f934843a17b7effc7&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:26.573324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:26.698231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba354bf1-11e6-4c66-86d1-abeacdccec15 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:26.702841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f14705-990f-452f-9780-62f5ba886f08 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:26.703134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f14705-990f-452f-9780-62f5ba886f08 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:26.703409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f14705-990f-452f-9780-62f5ba886f08 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:26.705128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 450/890] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:26.712135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:26.712632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:26.712771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:26.734465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:26.736508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:26.737640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:26.738634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:26.739177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:26.739803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:26.740615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:26.742439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:26.743947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:26.749724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 441/891] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:26 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:05:26.830868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 451/892] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:27.137360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:27.260258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2e3af-218b-4d28-9415-2dbd71437d8c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:27.299152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:27.299811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:27.300186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:27.300472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:27.312546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:27.319071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:27.320266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:27.321555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 442/893] 10.222.0.65 () {56 vars in 1120 bytes} [Mon Aug 29 21:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:27.328767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:27.343701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:27.460323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc962c70-ae65-4e7d-b790-865cb9aca5aa None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:27.498691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:27.499315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:27.499592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:27.499865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:27.500637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:27.500970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:27.502391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e3b2e-3140-4ecd-a252-acff3750cc5b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:27.512845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 452/894] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:05:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:27.519783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:27.588062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:27.638489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198163da-875a-4ac4-848b-add0d3004a0c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:27.672245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f50a1-801f-4259-b588-2d4613545255 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:27.672611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f50a1-801f-4259-b588-2d4613545255 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:27.672868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f50a1-801f-4259-b588-2d4613545255 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:27.674561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 453/895] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:27.680023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:27.680449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:27.680572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:27.680671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:27.681305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:27.681768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:27.682119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:27.698608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:27.700634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:27.705764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:27.706995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:27.720663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:27.721647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:27.723506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:27.723506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:27.723506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:27.723506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 443/896] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:27.729081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:27.806623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:27.821411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 454/897] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:27.898900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466b1df-4844-409b-ac61-e29b9eb1aafc None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:27.916411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:27.943144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:27.943777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:27.944098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:27.944375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:27.945041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:27.945476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:27.946907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ded1d8-16f6-49e9-a929-5f25ce7fc9b0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:27.963410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 444/898] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:27.969707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.092090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ae20e-e65f-469e-b5ef-8c1126162e8f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:28.128827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb79fb-dce0-4bc7-aff7-4d906aab4c62 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.129167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb79fb-dce0-4bc7-aff7-4d906aab4c62 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.129382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb79fb-dce0-4bc7-aff7-4d906aab4c62 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.131731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 445/899] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:28.134558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:28.135648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.136191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.136659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.137005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.137299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.137583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.147289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:28.149337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:28.166172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:28.167320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:28.168376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:28.169308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:28.169840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:28.171720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 455/900] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:05:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:28.180566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.260516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:28.279946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 446/901] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:28.279946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.340633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6ed5a-9398-4d6c-8b55-b6f3c6e9cf2c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:28.401941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:28.403265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.403773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.404879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.406191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:28.406781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:28.409343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:28.442493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 456/902] 10.222.0.65 () {56 vars in 1149 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:28.445486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.477677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:28.482803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28995ba9-e0b6-405f-b6c8-c5325dc823e0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:28.531185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:28.532835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.533360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.533836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.534771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:28.536498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:28.548210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:28.549295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:28.550151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:28.557840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 447/903] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:28.560766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.642216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da58b155-15b0-4aef-a15e-6c37fd79d61e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:28.674714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:28.699673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3bd95-785a-4f6a-bb3f-50857aeea036 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:28.705533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:28.706683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.707261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.707871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.741065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:28.742002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.742432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.742879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.743752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:28.744498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:28.746688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:28.758236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 448/904] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:28.760649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.808957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:28.822127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:28.824826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:28.920495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f0f0-e0e1-40c5-8b05-88448e80249f None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:28.965235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 457/905] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:05:28.965997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:28.971729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:28.972763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.973216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.973663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.974726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:28.975221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:28.977783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9637db8-0445-4f65-8f3f-8f19934ca4f4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:28.985480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 449/906] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:28.987211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2ce16-0d2a-45fc-86b9-b07ae8244ac4 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:28.987630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2ce16-0d2a-45fc-86b9-b07ae8244ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:28.988064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2ce16-0d2a-45fc-86b9-b07ae8244ac4 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:28.990371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 450/907] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:28.991763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2ce16-0d2a-45fc-86b9-b07ae8244ac4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.002926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:29.146071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b46ce-1fe4-4962-b7d8-e50b6ed7afda None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:29.148812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:29.168837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2ce16-0d2a-45fc-86b9-b07ae8244ac4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:29.199270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:29.199987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.200274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.200543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.201131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:29.204763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-bd46b682-e455-4818-80f4-791a6bb6fdd7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:29.217619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:29.219559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:29.221386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 458/908] 10.222.0.65 () {56 vars in 1132 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:29.222968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.222968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.222968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.236724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:29.237505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.237878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.238236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.245862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:29.247279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:29.255292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:29.257532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:29.283865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:29.291350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:29.292412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:29.292412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:29.292412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:29.292412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 451/909] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:29.292412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.364165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 459/910] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:29.364697 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.389538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:29.475840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6133e3-0be6-416e-b5a9-4f48873b0071 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:29.538963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:29.540300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.541016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.541656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.542608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:29.543069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:29.544664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:29.546589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1819d-161b-476d-b786-14d4f7fb3c68 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:29.559002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 452/911] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:29.560667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.578689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:29.602038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:29.603399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.603967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.604659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.605919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:29.606453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:29.608914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:29.634565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 460/912] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:29.637204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.728920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee439d62-1dda-481f-a97b-9b8f0df840f0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:29.748636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:29.784775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:29.785468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.785863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.786153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.786796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:29.787099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:29.789194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:29.799863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 453/913] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:29.802992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.815056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efae70-6440-4850-a8cc-5090ceaf9f0b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:29.889295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:29.890492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:29.890900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:29.892495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:29.905128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:29.909160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:29.931930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:29.932787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:29.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:29.934857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:29.941192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 461/914] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:29.941192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:29.980090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8424c-dbf0-4b9a-a068-36e23d896b02 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:30.031347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:30.032417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:30.032924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:30.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:30.034034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:30.035559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:30.046019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:30.047226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 21:05:30.048237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:30.049056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 454/915] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:30.049867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:30.106767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95854a-f18d-494e-bb6f-993d83c42b25 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:30.161025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:30.172764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:30.173787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:30.177141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:30.177141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:30.177141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:30.178678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:30.217013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0a52e-36f1-4615-a301-50dce7ccf01b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:30.272460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:30.273366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:30.273857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:30.274342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:30.275548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:30.275970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:30.278380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:30.283935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:30.314491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 455/916] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:30.314491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:30.320426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 462/917] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:30.322019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:30.533515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c05025-34f7-4711-93f5-7f64728ef402 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:30.548028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392605c8-0421-4302-8b28-06f817938a71 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:30.611714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:30.612617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:30.612905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:30.613243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:30.614298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:30.615109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:30.616855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:30.625250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 456/918] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:30.632410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:30.633224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:30.633402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:30.633802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:30.634249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:30.635810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] RBAC: Adding query filter params (role.id=e6123810ab85413f8e3c86d465369675, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3c2842cc986347dc802236ee27b9de50) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:30.637882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:30.638983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:30.649870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 463/919] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:30 2022] GET /identity/v3/role_assignments?user.id=3c2842cc986347dc802236ee27b9de50&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=e6123810ab85413f8e3c86d465369675 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:30.764466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:30.854369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d02c8-c6fa-4d68-94de-0a986e9f163b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:30.911387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:30.912198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:30.912579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:30.912956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:30.913697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:30.914121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:30.916028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459c9d1-92d0-4738-8a50-faa204c8e5d5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:30.923145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 457/920] 10.222.0.65 () {56 vars in 1114 bytes} [Mon Aug 29 21:05:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:30.931855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:31.015991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:31.114930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeec90-805c-4676-9381-84aa221721b8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:31.164087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240170e-9592-4cc5-8766-04a9bc17570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:31.165039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240170e-9592-4cc5-8766-04a9bc17570d None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:31.165496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240170e-9592-4cc5-8766-04a9bc17570d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:31.165946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240170e-9592-4cc5-8766-04a9bc17570d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:31.167141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240170e-9592-4cc5-8766-04a9bc17570d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:31.169484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240170e-9592-4cc5-8766-04a9bc17570d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:31.302095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 464/921] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:31.578300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68621e-28d7-459b-9f96-461d3529edc4 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:31.579597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68621e-28d7-459b-9f96-461d3529edc4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:31.579597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68621e-28d7-459b-9f96-461d3529edc4 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:31.581171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 458/922] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:31.588317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bfc1-f250-4db5-bfb0-6b44516175bd None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:31.588675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bfc1-f250-4db5-bfb0-6b44516175bd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:31.589009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32bfc1-f250-4db5-bfb0-6b44516175bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:31.610647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8f32bfc1-f250-4db5-bfb0-6b44516175bd None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:31.611954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8f32bfc1-f250-4db5-bfb0-6b44516175bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:31.727610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 465/923] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:32.110556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68621e-28d7-459b-9f96-461d3529edc4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:32.256430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68621e-28d7-459b-9f96-461d3529edc4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:32.288490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:32.302740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133d0fb-79f8-4c5c-b307-8094990bb2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:32.303796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d0fb-79f8-4c5c-b307-8094990bb2f5 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:32.304347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d0fb-79f8-4c5c-b307-8094990bb2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:32.304890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d0fb-79f8-4c5c-b307-8094990bb2f5 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:32.306340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133d0fb-79f8-4c5c-b307-8094990bb2f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:32.308779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133d0fb-79f8-4c5c-b307-8094990bb2f5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:32.395148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 459/924] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:32 2022] POST /identity/v3/services => generated 241 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:33.113901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe0052-8ec7-41fe-b5e5-ea99ebfeb901 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:33.114879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe0052-8ec7-41fe-b5e5-ea99ebfeb901 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:33.114879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe0052-8ec7-41fe-b5e5-ea99ebfeb901 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:33.116538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 466/925] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:33.122164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb276a9f-3ac3-47b4-be66-169f5d8a3671 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:33.122688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb276a9f-3ac3-47b4-be66-169f5d8a3671 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:33.122688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb276a9f-3ac3-47b4-be66-169f5d8a3671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:33.154488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-fb276a9f-3ac3-47b4-be66-169f5d8a3671 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:33.156074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-fb276a9f-3ac3-47b4-be66-169f5d8a3671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:33.278856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 460/926] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:33.624591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe0052-8ec7-41fe-b5e5-ea99ebfeb901 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:33.667935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:33.676381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf3a39-d0b4-40c4-bd07-b1fbbdeb1c72 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:33.677080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf3a39-d0b4-40c4-bd07-b1fbbdeb1c72 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:33.677563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf3a39-d0b4-40c4-bd07-b1fbbdeb1c72 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:33.679569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 461/927] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:33.685394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:33.685950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:33.686394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:33.711863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:33.714429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:33.789876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe0052-8ec7-41fe-b5e5-ea99ebfeb901 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:33.834985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 462/928] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:33.871352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:33.872510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:33.873024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:33.873523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:33.874695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:33.877459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b2d9997c-b49d-4c36-a4b8-d181eed4587f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:33.894605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:33.895759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:33.897325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 467/929] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:33.909589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:33.987378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ca69-d560-43c7-9eb7-1f061ff26e99 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:33.987906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ca69-d560-43c7-9eb7-1f061ff26e99 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:33.988415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ca69-d560-43c7-9eb7-1f061ff26e99 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:33.991261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 468/930] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:34.000860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:34.001840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:34.002367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:34.002877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:34.037973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:34.040269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:34.094412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b7a75-d29b-4e9e-91f9-853f706e21cc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:34.142667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:34.143352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:34.143567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:34.143849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:34.144589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:34.144914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:34.146318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:34.151976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 463/931] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:34.158481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:34.172095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 469/932] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:34.224056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:34.262033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:34.353527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcfc4a-f5c8-49b4-bcf5-9358021b4214 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:34.358686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:34.402773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f136193-1ddf-4a04-869d-4a60d3c95cc8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:34.422786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:34.423639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:34.424163 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:34.424643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:34.425774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:34.426216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:34.428505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:34.440315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 464/933] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:34.451449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:34.456809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:34.457494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:34.458022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:34.478728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:34.485473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:34.486723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:34.487647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.490933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 470/934] 10.222.0.65 () {56 vars in 1120 bytes} [Mon Aug 29 21:05:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:34.498778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:34.522859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:34.696596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:34.729413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d70b62-17d5-466e-82e1-85b54b7f6771 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:34.738177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9aa7b-a112-4af6-b0ce-b2be398eac59 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:34.786614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:34.787413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:34.787810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:34.788230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:34.788930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:34.789356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:34.792073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:34.806166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:34.807496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:34.808094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:34.808693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:34.809952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:34.811375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 465/935] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:05:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:34.812638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:34.812794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:34.818467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING keystone.server.flask.common [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 21:05:34.866703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:34.888491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.sql.core [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 21:05:34.888491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 21:05:34.888491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [parameters: {'id': 'a1158bda00b04692b1599ed69de2b39f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 21:05:34.888491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81875) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a1158bda00b04692b1599ed69de2b39f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a1158bda00b04692b1599ed69de2b39f. Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:05:34.912412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result.read() Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 21:05:34.913962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application yield resource Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise self.value Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application session.commit() Aug 29 21:05:34.915800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.session.flush() Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application compat.raise_( Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 21:05:34.919362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 21:05:34.923363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application result.read() Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:05:34.926896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [parameters: {'id': 'a1158bda00b04692b1599ed69de2b39f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:34.930348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:34.939631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a1158bda00b04692b1599ed69de2b39f. Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 471/936] 10.222.0.65 () {60 vars in 1140 bytes} [Mon Aug 29 21:05:34 2022] POST /identity/v3/projects => generated 262 bytes in 397 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:34.947266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:34.989926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e513289-716e-42e9-a86a-83c68269d0ba None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.043269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.044292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.044576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.044857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.060592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.062113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-88880831-1230-43c8-a43b-07b7085190fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:35.082593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:35.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 466/937] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc60b79-dc4a-4c64-8562-24d11d719387 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc60b79-dc4a-4c64-8562-24d11d719387 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc60b79-dc4a-4c64-8562-24d11d719387 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 467/938] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:35.092507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc60b79-dc4a-4c64-8562-24d11d719387 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:35.105977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39afd01e-5d96-4de4-9289-10c3aec22213 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.161561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.162517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.162955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.163388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.172703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.176100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.179538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-59686ee7-80be-4797-9df2-ffae810a0b3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:35.191142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:35.192158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 472/939] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.196837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.202141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.220254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:35.222730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:35.238435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:35.269019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc60b79-dc4a-4c64-8562-24d11d719387 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.314993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.315752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.316222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.316615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.317630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:35.317986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.319562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.346256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 468/940] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:35.347509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:35.351954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 473/941] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:35.357882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:35.527996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66ac2e-1f3e-4122-ae52-11483d5e4ccd None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.555246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184adc7e-4eb0-4b46-abcd-1748744e22be None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.582328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.584017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.584650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.585738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.588367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:35.589159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.591034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:35.593544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.601958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.603000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.603363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.603716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.614576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.617164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.625244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 469/942] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:35.635739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:35.637538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:35.638917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:35.643514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 474/943] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:05:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:35.649779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:35.652941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:35.727273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:35.807458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b80ce-5a00-43b5-8158-d436d4d53f5e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.862218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.862988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.863523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.863523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.865212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:35.865677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.868259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.896722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a356fcc-bac9-406c-bd2c-99f5620c05ca None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:35.899340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 470/944] 10.222.0.65 () {56 vars in 1149 bytes} [Mon Aug 29 21:05:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:35.914082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:35.959089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:35.960077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:35.960462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:35.960840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:35.961544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:35.963696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:35.987930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:35.989965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:35.991693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 475/945] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:36.001953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.015926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:36.095929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e50dce-b846-48ca-83e8-1ad8a2684d8c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.100031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:36.149511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.150511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.150892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.151326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.172682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.174512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:36.197160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:36.198306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:36.199328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:36.199328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:36.199328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:36.199328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 471/946] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:36.204424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.220281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb94c-000b-490e-b9c7-6e8e6709c300 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.312018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.313988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.314598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.315336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.316566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:36.317180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.320544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.338989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 476/947] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:36.348612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.391731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97648e77-7858-4e4b-b699-f77662d85b8b None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.419224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:36.457852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.459261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.459951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.460624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.461915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:36.462604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.465292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.489281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372e52-99aa-41e7-b63f-148399b8fe61 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.492457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 472/948] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:36.500016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.510551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:36.535845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.536680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.537004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.537338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.538243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:36.538846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.540841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.553846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 477/949] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:36.555357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.701149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cf3d7-a46c-44d0-887e-f8b58a55c26c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.703348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df82796-bc78-4712-9136-96a3c683a1e6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.747018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.748234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.748234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.748629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.749446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] RBAC: Adding query filter params (interface=public, service_id=6942f7979a8e4edb868008bd51e4357a, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:36.749890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.751381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.764994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 478/950] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/endpoints?service_id=6942f7979a8e4edb868008bd51e4357a&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:36.767584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.768988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.770117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.770704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.771302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.785087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.788222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:36.810036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:36.811436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:36.812818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 473/951] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:36.820148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:36.848841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:36.864899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:36.910780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2819f-8ae2-4df1-a2e2-a3c32c0d6622 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.949233 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56be65-9e04-4edc-b98a-942a9d2b5470 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:36.949543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0587be-b09f-4bdb-aedf-5571754c2a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:36.950239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0587be-b09f-4bdb-aedf-5571754c2a99 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.950582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0587be-b09f-4bdb-aedf-5571754c2a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.950925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0587be-b09f-4bdb-aedf-5571754c2a99 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.951682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0587be-b09f-4bdb-aedf-5571754c2a99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:36.953654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0587be-b09f-4bdb-aedf-5571754c2a99 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:36.966217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 479/952] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:05:36.967529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3cbb-409f-4812-bf54-78ca64716ec5 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.967816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3cbb-409f-4812-bf54-78ca64716ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.968121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa3cbb-409f-4812-bf54-78ca64716ec5 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.973979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 480/953] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:36.975380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:36.975668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:36.975996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:36.998980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:37.001295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:37.006254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:37.007209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:37.007688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:37.008194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:37.009205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:37.009834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:37.011711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:37.032791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 474/954] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:37.034117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:37.121601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 481/955] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:37.161949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:37.172333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3146c3-1476-4b4b-90ce-a4561ae44a27 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:37.216200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e60ba2-8b02-4079-aca4-9c07211e958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:37.218018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e60ba2-8b02-4079-aca4-9c07211e958d None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:37.218421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e60ba2-8b02-4079-aca4-9c07211e958d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:37.218825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e60ba2-8b02-4079-aca4-9c07211e958d None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3c2842cc986347dc802236ee27b9de50/roles/e6123810ab85413f8e3c86d465369675` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:37.222967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e60ba2-8b02-4079-aca4-9c07211e958d None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=3c2842cc986347dc802236ee27b9de50, role_id=e6123810ab85413f8e3c86d465369675)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:37.225092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e60ba2-8b02-4079-aca4-9c07211e958d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:37.409349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 475/956] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:05:37 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3c2842cc986347dc802236ee27b9de50/roles/e6123810ab85413f8e3c86d465369675 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:05:37.434124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:37.502305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:37.585662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d5f90-fd3f-43a4-a378-979fb47c36dc None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:37.640546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6cee1-27dc-4460-b063-9550a5caa3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:37.641884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6cee1-27dc-4460-b063-9550a5caa3d0 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:37.642523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6cee1-27dc-4460-b063-9550a5caa3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:37.643033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6cee1-27dc-4460-b063-9550a5caa3d0 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:37.644035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6cee1-27dc-4460-b063-9550a5caa3d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:37.646205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6cee1-27dc-4460-b063-9550a5caa3d0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:37.841259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 482/957] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:37 2022] POST /identity/v3/services => generated 242 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:38.900838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c302146-6e1e-4aa3-907c-9a6cb46051df None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:38.901410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c302146-6e1e-4aa3-907c-9a6cb46051df None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:38.901742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c302146-6e1e-4aa3-907c-9a6cb46051df None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:38.904665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 476/958] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:38.915056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0071c60-0012-4002-b75f-ed4ed04cea6b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:38.915612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0071c60-0012-4002-b75f-ed4ed04cea6b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:38.916027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0071c60-0012-4002-b75f-ed4ed04cea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:38.946446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b0071c60-0012-4002-b75f-ed4ed04cea6b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:38.948392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b0071c60-0012-4002-b75f-ed4ed04cea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:39.080388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 483/959] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:39.368297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f556-2abf-4aeb-8431-4e00d4ee613b None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:39.368673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f556-2abf-4aeb-8431-4e00d4ee613b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:39.368944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f556-2abf-4aeb-8431-4e00d4ee613b None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:39.370598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 477/960] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:39.376651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b719b1-6ca3-4c42-8dc8-e2dde9e9fe51 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:39.376938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b719b1-6ca3-4c42-8dc8-e2dde9e9fe51 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:39.377216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b719b1-6ca3-4c42-8dc8-e2dde9e9fe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:39.397517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-56b719b1-6ca3-4c42-8dc8-e2dde9e9fe51 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:39.399178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-56b719b1-6ca3-4c42-8dc8-e2dde9e9fe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:39.419760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:39.473590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 484/961] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:39.582517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f556-2abf-4aeb-8431-4e00d4ee613b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:39.674037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af46be-c4e2-46ac-a1e4-bf7df8592bb1 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:39.674235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af46be-c4e2-46ac-a1e4-bf7df8592bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:39.674508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af46be-c4e2-46ac-a1e4-bf7df8592bb1 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:39.676153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 485/962] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:39.689118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:39.694872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:39.694872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:39.710626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:39.712195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:39.772801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f556-2abf-4aeb-8431-4e00d4ee613b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:39.813069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 486/963] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:39.824612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:39.826775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:39.827693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:39.828115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:39.828570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:39.829497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:39.831067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:39.834079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING keystone.server.flask.common [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 21:05:39.888175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.sql.core [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 21:05:39.888175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 21:05:39.888175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [parameters: {'user_id': 'c6635d049696432f8b549d1fbc039335', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 21:05:39.888175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81874) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application result.read() Aug 29 21:05:39.915180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:05:39.917286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application yield resource Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application next(self.gen) Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise self.value Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application session.commit() Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 21:05:39.918889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self.session.flush() Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application compat.raise_( Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 21:05:39.926067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 21:05:39.929111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application conn.query(q) Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application result.read() Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 21:05:39.936188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c6635d049696432f8b549d1fbc039335', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:39.939092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 21:05:39.941164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 478/964] 10.222.0.65 () {60 vars in 1131 bytes} [Mon Aug 29 21:05:39 2022] POST /identity/v3/users => generated 160 bytes in 335 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:39.943445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:39.957034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:39.976540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:39.977684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722343c-e2fd-41fa-9fed-9cd85da61c63 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.026058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.027467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.027924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.028350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.029261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:40.032434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:40.048179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:40.056875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:40.065732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 487/965] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:40.067776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.105134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16275-d3d3-4adb-9e3d-fd78d0f95551 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.160888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.161569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.161842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.162115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.170626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:40.173002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:40.187491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:40.188545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 21:05:40.189633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:40.190200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cab01-f206-424a-bfd7-801abcc99891 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.190285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 479/966] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:40.190285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.233416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.234714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.235059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.235420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.236109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:40.236413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:40.238415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:40.253105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 488/967] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:40.254125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.271014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:40.320549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:40.350756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2e14a-48f0-4988-8c8e-f4ec87c6e815 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.398484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.399130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.399488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.399984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.403266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06285517-f9a5-4943-98f4-3dd536f6b47c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.414452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:40.421317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:40.423132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:40.424211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 480/968] 10.222.0.65 () {56 vars in 1120 bytes} [Mon Aug 29 21:05:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:40.425716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.456835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.457616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.457991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.458470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.459684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:40.459872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:40.462141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dee22-c91c-4f49-9560-adcbdbb99e38 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:40.516507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 489/969] 10.222.0.65 () {56 vars in 1101 bytes} [Mon Aug 29 21:05:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:40.517937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1ca0e-d254-4d07-9827-7b18978ca26b None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.518250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1ca0e-d254-4d07-9827-7b18978ca26b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.518535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1ca0e-d254-4d07-9827-7b18978ca26b None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.520000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 490/970] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:40.520924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1ca0e-d254-4d07-9827-7b18978ca26b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.607730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1b30e-3505-474e-9de4-935fa548fc75 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.641158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:40.657309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40732817-5db8-4905-970b-c4998140b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.659200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40732817-5db8-4905-970b-c4998140b111 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.659200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40732817-5db8-4905-970b-c4998140b111 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.659583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40732817-5db8-4905-970b-c4998140b111 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.660732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40732817-5db8-4905-970b-c4998140b111 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:40.661234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40732817-5db8-4905-970b-c4998140b111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:40.663761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40732817-5db8-4905-970b-c4998140b111 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:40.666420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:40.679851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 481/971] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:40.681862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1ca0e-d254-4d07-9827-7b18978ca26b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.682523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.682994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.683470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.717930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:40.720849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:40.733028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:40.733787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:40.734097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:40.734374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:40.735064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:40.735367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:40.736830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:40.754817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 491/972] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:05:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:40.756931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.860543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 482/973] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:40.860791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:40.938776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2653e50-fe0b-4e70-ba31-276aef6ec27c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:40.994303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:41.002861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.003914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.004412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.004872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.010246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.010246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.052813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0718ba-ecf1-4595-acb2-4c7b796baa74 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.100954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:41.124014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 492/974] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:41.124801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.124801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.124801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.124801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.138894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.141677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.150835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:41.173725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:41.175535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:41.177271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:41.177271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:41.177271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:41.183750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 483/975] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:41.191495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.364495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c80341-87f4-4c97-afbe-369c72796456 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.375698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406ed3c-39be-4a64-8366-c230229eb817 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.421878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.422872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.423343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.423846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.424878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:41.425349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.426790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.427241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.427503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.427763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.428089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.428618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.430072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:41.441093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:41.442287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:41.444958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 493/976] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:41.450871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.458344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 484/977] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:41.461774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.535740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:41.547125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:41.612810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c4c83-cb27-4efc-ab6a-ddfc8bb73986 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.658529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.658849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4bf6d-8bf9-4a40-b2fe-6513ba888c1e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.659133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.659443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.659738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.660417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:41.660720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.662288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.673250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 494/978] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:41.674584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.721414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.722046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.722478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.722901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.736581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.739377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:41.762660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:41.764566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:41.765702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 485/979] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:41.774441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.818063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548bb19-c4bf-4b22-9566-7dbb81801bfb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.843718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:41.894576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.895535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.895981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.896392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:41.897356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:41.897831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:41.900312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:41.911495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 495/980] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:41.918171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:41.946634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:41.951117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523787e9-87cd-4a46-b779-32e7c191a0e7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:41.998674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:41.999340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:41.999620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:41.999964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:42.000724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:42.001019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:42.002845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:42.036175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 486/981] 10.222.0.65 () {56 vars in 1149 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:42.037716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:42.080417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696a45a-3a39-4892-949c-7b3023e0e6f7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:42.148446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:42.149349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:42.149715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:42.150155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:42.151441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] RBAC: Adding query filter params (interface=public, service_id=c523c0c1d93f49fba3bab8088c03688a, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:42.151990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:42.154585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:42.166009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 496/982] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:41 2022] GET /identity/v3/endpoints?service_id=c523c0c1d93f49fba3bab8088c03688a&interface=public®ion_id=RegionOne => generated 194 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:42.174225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:42.219075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc240d6-b045-4491-b82b-b9511ab3f76c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:42.268881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:42.295324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:42.296311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:42.296753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:42.297231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:42.339210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:42.388766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0635f-67b2-48e4-bcc8-6ab17103a3ad None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:42.410717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:42.413492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:42.438517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:42.439344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:42.439816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:42.440408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:42.441400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:42.443624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:42.455588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 497/983] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:42 2022] GET /identity/v3/services => generated 1727 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:05:42.599985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 487/984] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:42 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:05:42.606883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:42.817276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e34bef-24ba-4811-b489-8c46dda2c8cb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:42.824446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:42.881628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:42.882781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:42.883255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:42.883737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:42.903598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:42.906386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:42.935707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:42.938672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:42.940429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:42.940429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:42.940429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:42.940429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 498/985] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:42.946866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:43.072553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:43.166941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290174-9e2c-482d-84be-5785afdbca2a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:43.235566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:43.237901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:43.238671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:43.239404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:43.240813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:43.241568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:43.244138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:43.269973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 488/986] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:43.278362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:43.488248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683bc4-a4aa-40f2-9957-18bdc7b02920 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:43.555014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:43.555827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:43.556373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:43.556867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:43.569803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:43.572620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7e52f74e-c20f-4fe0-99e6-a3d5b2dd58ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:43.596856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:43.598308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:43.599623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 499/987] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:43.612369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:43.772372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffc261-b8ab-401f-be31-eea1ea30ba1c None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:43.772924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffc261-b8ab-401f-be31-eea1ea30ba1c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:43.773249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffc261-b8ab-401f-be31-eea1ea30ba1c None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:43.775759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 500/988] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:43.785365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:43.785989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:43.786784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:43.815721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:43.821630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:43.843035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d80cd-3fd0-4626-b64b-f354eb85d724 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:43.850081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:43.875161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:43.904781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:43.905470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:43.905755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:43.906032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:43.906773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:43.907083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:43.908742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:43.924138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 489/989] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:43.933758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:43.950100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 501/990] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:44.136884 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2e67c-67b4-4e1f-aab7-4790924f4a97 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:44.196411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:44.197405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:44.197986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:44.198720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:44.200563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] RBAC: Adding query filter params (role.id=e6123810ab85413f8e3c86d465369675, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3c2842cc986347dc802236ee27b9de50) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:44.201686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:44.205388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:44.216283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 490/991] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:43 2022] GET /identity/v3/role_assignments?user.id=3c2842cc986347dc802236ee27b9de50&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=e6123810ab85413f8e3c86d465369675 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:44.246395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:44.411867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d9303-dc3c-427e-8cc4-cabdce4c5ee2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:44.458256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:44.458914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:44.459238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:44.459639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:44.460315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:44.461881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:44.473493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:44.475478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:44.477474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 502/992] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:44.481828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:44.543058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:44.618992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065faf4-dc14-4205-a70c-960a847460bf None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:44.657098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:44.658024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:44.658546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:44.659027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:44.659868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:44.660440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:44.662480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d2d4-1be9-4412-89b7-8b65a476e544 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:44.668938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 491/993] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:44.675157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:44.691266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:44.809472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38641c0-9557-4095-bfa0-2ff51591f0d1 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:44.840130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fbb4a-9751-456f-968d-b5645db7929e None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:44.840579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fbb4a-9751-456f-968d-b5645db7929e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:44.840994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fbb4a-9751-456f-968d-b5645db7929e None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:44.843353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 492/994] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:44.849426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:44.849713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:44.849994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:44.855836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:44.856980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:44.857267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:44.858474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:44.859282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:44.859501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:44.860940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:44.867063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 503/995] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:44.869226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:44.870441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:44.873029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:44.959456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 493/996] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:44 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:44.985354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:45.029951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b1490-e967-4c58-86fa-f71de1137f4b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:45.072996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:45.073619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:45.073924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:45.074173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:45.074952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] RBAC: Adding query filter params (interface=admin, service_id=6942f7979a8e4edb868008bd51e4357a, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:45.075265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:45.077358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:45.083935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 504/997] 10.222.0.65 () {56 vars in 1249 bytes} [Mon Aug 29 21:05:44 2022] GET /identity/v3/endpoints?service_id=6942f7979a8e4edb868008bd51e4357a&interface=admin®ion_id=RegionOne => generated 193 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:45.089379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:45.135100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:45.262569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4582d-f32b-46dc-bae7-4216f24a6fae None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:45.330148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:45.331172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:45.331652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:45.332243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:45.333328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:45.335863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:45.353346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 494/998] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:45 2022] GET /identity/v3/services => generated 1727 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:05:45.353556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:45.553608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af19473-a4d4-4548-9638-e9f61736ebba None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:45.605147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:45.605965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:45.606387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:45.606792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:45.607519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:45.609585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:45.620626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:45.621799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:45.622767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 505/999] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:45.633962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:45.790947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:45.796654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c49bc5-b48b-461f-a4ff-698db0f49064 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:45.854539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:45.855360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:45.855739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:45.856144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:45.857034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:45.857464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:45.860274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:45.869959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 495/1000] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:45.876219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:45.879875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:46.019250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0942-f776-4fb4-aec7-a48a15dbfd7b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:46.078439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:46.079493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:46.080033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:46.080534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:46.081713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:46.082249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:46.084983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5b21d-c806-4990-9204-0495c2ff7506 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:46.100035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 506/1001] 10.222.0.65 () {56 vars in 1112 bytes} [Mon Aug 29 21:05:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:46.103115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:46.277432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b621a22-bd3a-4449-b9c3-7c07d751baf6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:46.330084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c405f-8b94-41d9-8701-eaf3fc664376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:46.330912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c405f-8b94-41d9-8701-eaf3fc664376 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:46.331352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c405f-8b94-41d9-8701-eaf3fc664376 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:46.331792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c405f-8b94-41d9-8701-eaf3fc664376 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:46.332867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c405f-8b94-41d9-8701-eaf3fc664376 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:46.335551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c405f-8b94-41d9-8701-eaf3fc664376 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:46.472038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 496/1002] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:46.687210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185bafa-5214-471f-ab41-9e4e5184f08e None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:46.688123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185bafa-5214-471f-ab41-9e4e5184f08e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:46.688123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185bafa-5214-471f-ab41-9e4e5184f08e None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:46.689522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 507/1003] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:46.695504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:46.703792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:46.703792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:46.717059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:46.718981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:46.739314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:46.855478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 497/1004] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:46 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:47.341705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185bafa-5214-471f-ab41-9e4e5184f08e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:47.431052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:47.476006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185bafa-5214-471f-ab41-9e4e5184f08e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:47.518758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:47.519526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:47.519811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:47.520116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] PATH_INFO: `/v3/services/image` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:47.520708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:47.522101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6c86012d-15d0-4764-8a43-8d0803d15d84 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:47.532701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:47.533775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:47.534818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:47.539338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 508/1005] 10.222.0.65 () {56 vars in 1105 bytes} [Mon Aug 29 21:05:47 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:47.540319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:47.632154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68259f-330e-4438-8a7f-083d122ffbfa None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:47.632607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68259f-330e-4438-8a7f-083d122ffbfa None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:47.633055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68259f-330e-4438-8a7f-083d122ffbfa None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:47.635001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 509/1006] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:47.641124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:47.641461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:47.641711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:47.661705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:47.663124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:47.674523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156028f4-df0f-4994-8f50-52e78cb8b1cf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:47.729908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:47.730658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:47.730819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:47.731045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:47.731704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:47.732173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:47.734280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:47.740903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 498/1007] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:47.746844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:47.776028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 510/1008] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:47 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:47.776874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:47.939849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b8a55-be97-4abf-8357-2db87a9ab7e0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:48.007572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:48.008488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:48.008945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:48.009381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:48.010440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] RBAC: Adding query filter params (type=image) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:48.010931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:48.013448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:48.023228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 499/1009] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:48.180411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:48.269973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:48.347704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8819c-72fa-4729-8cf5-bd37ee620aa0 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:48.414087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:48.415191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:48.415690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:48.416217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:48.417395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:48.420066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:48.438194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:48.439311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 21:05:48.440602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:48.441736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 511/1010] 10.222.0.65 () {56 vars in 1126 bytes} [Mon Aug 29 21:05:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:48.449177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:48.583404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:48.615582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6583f57-a71e-43d1-94b6-9ae183d17358 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:48.658792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:48.659629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:48.659935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:48.660213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:48.660863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:48.661166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:48.662626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:48.670003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 500/1011] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:48.677367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:48.804256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181fa3c-5377-423f-b5fa-b1a205936ac5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:48.836891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:48.850998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:48.851923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:48.853935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:48.853935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:48.853935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:48.853935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:48.855116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ba6af-573f-42b9-b224-b55a844095fe None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:48.861065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 512/1012] 10.222.0.65 () {56 vars in 1122 bytes} [Mon Aug 29 21:05:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:48.867040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:48.983533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97124daf-844a-4113-9a83-a3ef8b86565d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:49.028889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:49.029794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:49.030175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:49.030627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:49.031669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:49.033477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:49.034459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef45cc4-3bbb-4dd0-8c1e-1944af24bba5 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:49.034750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef45cc4-3bbb-4dd0-8c1e-1944af24bba5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:49.035049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef45cc4-3bbb-4dd0-8c1e-1944af24bba5 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:49.036666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 513/1013] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:49.042295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:49.042841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:49.043096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:49.065304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:49.066500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:49.134305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:49.142691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 501/1014] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:49.162001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 514/1015] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:49 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:49.447526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:49.581866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bcd6d-0e70-4d19-921c-ebd198617408 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:49.647113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:49.648260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:49.649910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:49.649910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:49.672277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:49.683043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:49.684119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:49.685091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:49.688711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 502/1016] 10.222.0.65 () {56 vars in 1120 bytes} [Mon Aug 29 21:05:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:49.694709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:49.877986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805981a-5d41-4976-b51f-ab6daba63a2d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:49.904896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:49.939322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:49.940391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:49.940896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:49.941387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:49.942579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:49.943140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:49.946410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:49.968058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 515/1017] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:05:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:49.970384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:50.062342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:50.129956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb99f-c862-4c81-985b-c11abe5d693d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:50.203136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:50.204231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.204720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.205215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.226065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:50.230274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-af2a0e68-4e4a-4373-a514-3902a79dd40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:50.258567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:50.259631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:50.260602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:50.260602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:50.260602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:50.268779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 503/1018] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:50.276713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:50.284838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbe5c9-98cc-441b-a31c-d6107d73ff90 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.285101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbe5c9-98cc-441b-a31c-d6107d73ff90 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.285383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbe5c9-98cc-441b-a31c-d6107d73ff90 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.286810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 504/1019] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:50.294844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.295178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.295425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.322225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:50.324426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:50.431459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50195a-c529-49bd-bf98-cf3e1fa4c65e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:50.436966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 505/1020] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:50 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:50.477449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:50.478074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.478353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.478619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.479274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:50.479577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:50.481323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:50.496410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 516/1021] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:50.503195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:50.554322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:50.625693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9d013-52a3-434a-a61f-87f3198fd8e7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:50.671033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:50.671754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.672136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.672610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.682191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:50.683730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:50.698275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:50.700033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:50.702301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:50.707933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 506/1022] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:05:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:50.709043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:50.717685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:50.769118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:50.898808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:50.904916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb228b-6943-4cf1-b55f-3543e5d28144 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:50.908668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ef85c-4d76-4d5a-89ba-cf29e43ab030 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:50.960691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb9ab9-2352-490d-bf72-7f6bfd390d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:50.961333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb9ab9-2352-490d-bf72-7f6bfd390d88 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.961755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb9ab9-2352-490d-bf72-7f6bfd390d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.961934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:50.962120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb9ab9-2352-490d-bf72-7f6bfd390d88 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.962486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:50.962778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb9ab9-2352-490d-bf72-7f6bfd390d88 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:50.962936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:50.963074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:50.963832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:50.964159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:50.964341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb9ab9-2352-490d-bf72-7f6bfd390d88 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:50.965684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:50.994675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 517/1023] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:05:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:50.996016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:51.049599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 507/1024] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:50 2022] POST /identity/v3/services => generated 242 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:05:51.164780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:51.175461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133589-1338-4586-b6a4-5c3a9b7dc868 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:51.235949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:51.237006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:51.237474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:51.237988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:51.361096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:51.362640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:51.437219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d170-abfe-40fb-a1db-06eca3c26444 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:51.437472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d170-abfe-40fb-a1db-06eca3c26444 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:51.437758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d170-abfe-40fb-a1db-06eca3c26444 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:51.439362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 508/1025] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:51.445290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:51.445552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:51.445829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:51.466538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:51.467961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:51.505628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:51.523124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 518/1026] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:05:50 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:05:51.532225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:51.571053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 509/1027] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:51.675633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe72f9b-845d-415f-92b6-87feacd690ac None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:51.733282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:51.734121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:51.734531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:51.734930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:51.749654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:51.751208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:51.770499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:51.772872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:51.774694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:51.774694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:51.774694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:51.776306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 519/1028] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:51.777558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:51.892055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:51.919381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:51.931162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089428f-88cc-4081-a811-9a499b4107aa None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:51.962439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:51.972540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:51.973500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:51.973956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:51.974450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:51.975325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:51.975634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:51.977342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:51.991775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 510/1029] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:51.997159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:52.049770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555236f3-8c55-42e6-9968-7de141484ddc None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:52.095134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:52.096059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:52.096186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:52.096361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:52.097698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:52.099826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:52.102737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 520/1030] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:52.110595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:52.134525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1de08-f513-4ae6-be25-51b83808e805 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:52.181010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:52.181663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:52.181942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:52.182284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:52.193622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:52.195376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:52.208957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:52.210153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:52.211491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:52.215546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 511/1031] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:52.220658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:52.237309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:52.271139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922f158-4b87-4e2f-9cb9-e0601abd6172 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:52.286564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:52.334720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3eb49-caee-4285-8e29-675b9b7a9abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:52.336025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3eb49-caee-4285-8e29-675b9b7a9abb None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:52.336755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3eb49-caee-4285-8e29-675b9b7a9abb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:52.337427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3eb49-caee-4285-8e29-675b9b7a9abb None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:52.338199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3eb49-caee-4285-8e29-675b9b7a9abb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:52.340391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3eb49-caee-4285-8e29-675b9b7a9abb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:52.367922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee611208-3c6b-4b70-b09d-131950cb9d36 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:52.426523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:52.427276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:52.427649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:52.428157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:52.429288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:52.429762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:52.431360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:52.446159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 512/1032] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:52.457247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:52.488898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 521/1033] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:05:52 2022] POST /identity/v3/projects => generated 311 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:52.581436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:52.639931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96323b12-6b19-4db7-8593-fa384096539a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:52.700660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:52.702488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:52.703231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:52.703653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:52.704579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] RBAC: Adding query filter params (role.id=e6123810ab85413f8e3c86d465369675, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3e5014eab4c7479f9ee493e31c91b3f4) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:52.704923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:52.706562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608ff38-b7fb-4791-8904-9b8dd5971bbe None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:52.714556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 513/1034] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:05:52 2022] GET /identity/v3/role_assignments?user.id=3e5014eab4c7479f9ee493e31c91b3f4&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=e6123810ab85413f8e3c86d465369675 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:53.685803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5cff8-6234-487f-8504-7724e25d4089 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:53.686592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5cff8-6234-487f-8504-7724e25d4089 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:53.686592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5cff8-6234-487f-8504-7724e25d4089 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:53.688015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 522/1035] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:53.696822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:53.697600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:53.697600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:53.721329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:53.722569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:53.964276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 514/1036] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:54.460750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5cff8-6234-487f-8504-7724e25d4089 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:54.628684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5cff8-6234-487f-8504-7724e25d4089 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:54.693945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:54.695075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:54.697069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:54.697069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] PATH_INFO: `/v3/services/image` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:54.697399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:54.702463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-2c7062a5-e986-418f-8845-b3d10966f500 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:54.721194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:54.722273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:54.723231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:54.728306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 523/1037] 10.222.0.65 () {56 vars in 1105 bytes} [Mon Aug 29 21:05:54 2022] GET /identity/v3/services/image => generated 86 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:54.734454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:54.856816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21b6dd-3f0a-4b72-b6f0-4abbcd24224d None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:54.857964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21b6dd-3f0a-4b72-b6f0-4abbcd24224d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:54.858522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21b6dd-3f0a-4b72-b6f0-4abbcd24224d None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:54.861157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 524/1038] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:54.868320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:54.869199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:54.869199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:54.872361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:54.892409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:54.894789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:54.927344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:54.940486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9472da2-97ad-4b4c-8964-bcaceb973834 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:54.989369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:54.989930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:54.990210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:54.990479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:54.991118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] RBAC: Adding query filter params (name=image) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:54.991418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:54.992842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:54.999175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 515/1039] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:55.002977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 525/1040] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:55.005357 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:55.165230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c85b2-6a94-4383-beef-cea02f7bba0c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:55.208490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:55.217979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:55.218936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.219346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.241046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.241046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:55.241046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:55.241046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b7e79-1623-42b5-a747-4d93219b55f5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:55.241046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 516/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:55.241547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:55.265316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714967b6-7b64-443d-aa58-1ed29946b333 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.265585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714967b6-7b64-443d-aa58-1ed29946b333 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.265950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714967b6-7b64-443d-aa58-1ed29946b333 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.267689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 517/1042] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:55.274461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.274879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.275048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.300431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:55.301669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:55.382268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ed6b5-a605-4084-831d-8658a50dbef4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:55.389791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 518/1043] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:55.390493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:55.426129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:55.426811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.427125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.427609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.428554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] RBAC: Adding query filter params (interface=public, service_id=28411028f6714879b6bf891a7fdfd754, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:55.429068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:55.431119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:55.442140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 526/1044] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:05:55 2022] GET /identity/v3/endpoints?service_id=28411028f6714879b6bf891a7fdfd754&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:55.445723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:55.482757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:55.542802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2ced0-3278-415c-b3f0-fe991b9dd07c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:55.579088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05956e-b22b-4699-a3d9-57e3b2dfb7b2 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:55.583262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:55.598060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:55.599054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.599537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.599907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.601131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:55.602845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:55.608487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 519/1045] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:55.611146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:55.628432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:55.629413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.629900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.630362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.631381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:55.633319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:55.642186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 527/1046] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:05:55 2022] GET /identity/v3/services => generated 1957 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:05:55.706985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:55.766193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ab4d6f-abd9-461b-8594-742780902d8a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:55.836203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:55.837208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.837559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.837843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.838521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:55.840028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:55.840655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:55.869956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7883c85-84d6-4f23-8592-f8136bb3db1f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:55.915052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:55.916231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:55.916707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:55.917103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:55.930459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:55.936701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:55.938738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:55.940543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 528/1047] 10.222.0.65 () {56 vars in 1120 bytes} [Mon Aug 29 21:05:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:55.946300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:56.090505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e7eca9-ba15-4b6f-bbb1-2d35a6a5541b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:56.094594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 520/1048] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:05:55 2022] POST /identity/v3/users => generated 285 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:05:56.140011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:56.140761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:56.141105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:56.141439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:56.142326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:56.142659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:56.144420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:56.156484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 529/1049] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:05:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:56.163439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:56.236055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:56.324925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f64123-f4f3-48a4-9c53-f85f598436dc None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:56.382512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:56.382942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:56.383620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:56.384307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:56.405130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:56.406635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:56.425466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:56.426655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:56.427796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:56.427796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:56.427796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:56.427796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 521/1050] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:56.433974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:56.448181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:56.551497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e74b1-057f-4c61-9584-5812686df163 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:56.596153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:56.596766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:56.597043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:56.597411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:56.598001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:56.598305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:56.599981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:56.615741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 530/1051] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:56.622059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:56.813532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:56.819077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb857ad-473d-4880-b9d6-5864f6903b76 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:56.873739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:56.874579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:56.874990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:56.875399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:56.885119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:56.886806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:05:56.901292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:56.904396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:56.906301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 522/1052] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:05:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:56.910312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:57.002426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:57.045760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f9604-5f11-4b41-864d-5ecb48852f56 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:57.096827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:57.097657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:57.098308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:57.098482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:57.103210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:57.103210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:57.103210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:57.137694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 531/1053] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:05:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:57.145014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:57.363354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168c76d-4275-44dd-ba61-a1efd730cb08 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:57.428282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:57.428930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:57.429206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:57.429481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:57.449863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:57.453010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:05:57.478560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:57.479603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:05:57.481786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:05:57.481786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:05:57.481786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:57.481786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 523/1054] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:57.491993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:57.701889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6b991-75f6-4419-bd4e-465dde5d5467 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:57.742079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:57.769032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:57.770004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:57.770446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:57.770984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:57.772232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:57.772841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:57.774493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:57.795178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 532/1055] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:57.796708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:57.813123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:57.936084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fe150-8243-4e21-ab61-fa7f6bdb6ede None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:57.986024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:57.986923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:57.987411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:57.987937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:57.998026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:57.999515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-536830bc-a7f9-4238-aef9-c21cdd2ababf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:05:58.015686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:58.024861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:58.026923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 524/1056] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:05:58.031321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:58.034193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6f8c2-af4a-4f3e-a487-da1ccca15c84 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.034643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6f8c2-af4a-4f3e-a487-da1ccca15c84 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.035085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6f8c2-af4a-4f3e-a487-da1ccca15c84 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.037458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 525/1057] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:05:58.046240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.046699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.047147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.076423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:58.078290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:58.192101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:58.204011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 526/1058] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:05:58.223465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620899-e690-4463-aa66-10ea91c15529 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:58.273877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:58.274458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.274854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.275194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.276278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:58.276693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:58.279025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:58.296711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 533/1059] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:58.306330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:58.514480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4705509-6333-4a88-9bb4-d155d51b8da8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:58.608338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e88321-2287-475d-b504-0973f700eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:58.609594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e88321-2287-475d-b504-0973f700eb71 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.610189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e88321-2287-475d-b504-0973f700eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.610842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e88321-2287-475d-b504-0973f700eb71 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3e5014eab4c7479f9ee493e31c91b3f4/roles/e6123810ab85413f8e3c86d465369675` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.614927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e88321-2287-475d-b504-0973f700eb71 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=3e5014eab4c7479f9ee493e31c91b3f4, role_id=e6123810ab85413f8e3c86d465369675)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:58.617594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e88321-2287-475d-b504-0973f700eb71 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:58.651842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:58.697135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 527/1060] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:05:58 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/3e5014eab4c7479f9ee493e31c91b3f4/roles/e6123810ab85413f8e3c86d465369675 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:05:58.730942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b8eed-207f-4126-826d-bff6cd88e77a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.731498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b8eed-207f-4126-826d-bff6cd88e77a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.732501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b8eed-207f-4126-826d-bff6cd88e77a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.735556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 528/1061] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:05:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:05:58.743606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:58.745683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.746195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.747558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.786321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:05:58.788181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:05:58.861402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b111-180a-49fd-8ca9-9048f9fa196c None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:58.899578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:58.926314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 529/1062] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:05:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:05:58.954221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:58.955078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:58.955504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:58.956042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] PATH_INFO: `/v3/services/image` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:58.957099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:58.959129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:58.987304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:58.989929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:05:58.991384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 534/1063] 10.222.0.65 () {56 vars in 1105 bytes} [Mon Aug 29 21:05:58 2022] GET /identity/v3/services/image => generated 86 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:58.998961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:59.222419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3938bc5-f6c3-4b8c-841a-e9ccedb8b3af None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:59.266408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:59.267028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:59.267316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:59.267588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:59.268389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] RBAC: Adding query filter params (name=image) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:59.268665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:59.270184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:59.275727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 530/1064] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:05:59.282246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:59.309173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:59.396040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:59.457684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b44ca-adbf-4874-8edc-d570f7aea755 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:59.466663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3212a944-22ea-419c-b3d1-42ddbe201583 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:59.468700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:59.505390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:59.506319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:59.506744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:59.507197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:59.508158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] RBAC: Adding query filter params (type=image) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:59.508940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:59.512490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:59.530578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 535/1065] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:05:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:59.530875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:59.530875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:59.530875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:59.530875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:59.532575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:59.545667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:05:59.551266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:05:59.552552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:05:59.553431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:05:59.557877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 531/1066] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:05:59.558744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:59.690012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874693c3-0dc8-45e1-afca-e6d79cf42e19 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:59.735643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704bc78b-24be-4a09-a5bb-ff94afa9742e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:05:59.747581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:59.748689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:59.749191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:59.749674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:59.750945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:59.753641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:59.754774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:05:59.780376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:05:59.782319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:05:59.782319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:05:59.782319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:05:59.782319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:05:59.782726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:05:59.784853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:05:59.795616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 532/1067] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:05:59.802358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:05:59.853623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 536/1068] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:05:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:05:59.982696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411bac1-3598-48b0-89ef-00d8220202e3 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:00.068391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:00.070678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:00.072064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:00.072064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] PATH_INFO: `/v3/users/d3a6591c8d6147bd91662eac7030e515` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:00.092316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] RBAC: Authorizing `identity:get_user(user_id=d3a6591c8d6147bd91662eac7030e515)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:00.093797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:00.095636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 533/1069] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:05:59 2022] GET /identity/v3/users/d3a6591c8d6147bd91662eac7030e515 => generated 285 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:00.104103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:00.154051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:00.252926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd618d62-76af-433d-ab95-4f708ae4a27d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:00.292505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:00.293496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:00.293934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:00.294372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:00.378014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:00.379792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:00.413481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:00.531861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 537/1070] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:00.541337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:00.600647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:00.721935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ded864-f92e-470c-bab5-9651656abcb0 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:00.777962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c6cf-c4a4-4913-9bd8-c5e8e20b7d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:00.778948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c6cf-c4a4-4913-9bd8-c5e8e20b7d2f None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:00.779374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c6cf-c4a4-4913-9bd8-c5e8e20b7d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:00.779790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c6cf-c4a4-4913-9bd8-c5e8e20b7d2f None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:00.781804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208c6cf-c4a4-4913-9bd8-c5e8e20b7d2f None admin] RBAC: Authorizing `identity:get_project(project_id=c6b8da3773674e46a0132a5181eef3d5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:00.784317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208c6cf-c4a4-4913-9bd8-c5e8e20b7d2f None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:00.793091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 534/1071] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:00 2022] GET /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5 => generated 311 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:00.795781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:00.932067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875bd7-758c-49a0-90be-609f9609b3d6 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:00.966255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:00.976636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:00.980652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:00.980652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:00.980652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:00.980652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=c6b8da3773674e46a0132a5181eef3d5, user.id=d3a6591c8d6147bd91662eac7030e515) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:00.980652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:00.981862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c2c22-66dc-441a-9e85-82ce8c13dbf9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:00.992407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 538/1072] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:00 2022] GET /identity/v3/role_assignments?user.id=d3a6591c8d6147bd91662eac7030e515&scope.project.id=c6b8da3773674e46a0132a5181eef3d5&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:01.442507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc60a0f-f724-44f8-a551-d150a03e1edb None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:01.443244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc60a0f-f724-44f8-a551-d150a03e1edb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:01.443244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc60a0f-f724-44f8-a551-d150a03e1edb None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:01.444734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 535/1073] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:01.454243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:01.454784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:01.455163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:01.487160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:01.489236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:01.623668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 539/1074] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:01.922817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc60a0f-f724-44f8-a551-d150a03e1edb None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:02.056339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:02.070275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc60a0f-f724-44f8-a551-d150a03e1edb None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:02.117677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:02.118336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.118658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.118904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:02.137370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:02.142359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:02.143502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 21:06:02.144803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:02.148274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 536/1075] 10.222.0.65 () {56 vars in 1120 bytes} [Mon Aug 29 21:06:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:02.151636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:02.268618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2eeaf7-f37a-4f82-b06a-c65a8e20e49d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:02.294351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:02.308109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:02.311651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.311651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.311651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:02.311651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:02.311651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:02.312252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b898e-9110-4a96-bc39-7057acc1c1f3 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:02.322278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 540/1076] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:06:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:02.329190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:02.442170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb75b626-644d-45b1-8d82-f4985ce7865d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:02.453208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0a67-e05c-455e-bea7-2820f9aceb6d None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.453723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0a67-e05c-455e-bea7-2820f9aceb6d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.454242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb0a67-e05c-455e-bea7-2820f9aceb6d None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:02.463120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 541/1077] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:02.465898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.466415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.466915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:02.481633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:02.482308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.482553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.482822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:02.494089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:02.495550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:02.498532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:02.500628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:06:02.511177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:02.512510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:06:02.513911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:06:02.513911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:06:02.513911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:02.516804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 537/1078] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:02.518983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:02.540711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:02.631456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 542/1079] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:02.717984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72516b6-7c0f-4c34-9b80-f76185893755 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:02.776317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:02.777002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.777383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.777661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:02.779061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:02.779675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:02.781131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:02.798250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 538/1080] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:02.804945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:02.944528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8127d1-4f5c-4e68-9aa1-aef90bb221d7 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:02.966472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:02.981730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:02.994997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:02.995691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:02.996001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:02.996262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.007994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.009706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:06:03.032035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:03.033325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 21:06:03.034385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:06:03.037537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 543/1081] 10.222.0.65 () {56 vars in 1093 bytes} [Mon Aug 29 21:06:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:03.040484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.085823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:03.133044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c63045-7144-4ceb-b6d5-1f5d0d48f6a5 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:03.174437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52c139-a11f-4387-89df-33969c9ae2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:03.175384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c139-a11f-4387-89df-33969c9ae2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.175724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c139-a11f-4387-89df-33969c9ae2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.176152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52c139-a11f-4387-89df-33969c9ae2b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.180463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0def1cc-6bc1-40bc-8508-f4f33a8f1a17 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:03.222636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:03.224124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.224882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.225601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.227433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:03.228257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.231806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.263171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52c139-a11f-4387-89df-33969c9ae2b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.263545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 544/1082] 10.222.0.65 () {56 vars in 1133 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:03.265340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52c139-a11f-4387-89df-33969c9ae2b5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.269921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.318835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:03.353905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:03.389714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 539/1083] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:06:03.391025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749607c-6b3a-49f1-8797-40128fe32882 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.391311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749607c-6b3a-49f1-8797-40128fe32882 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.391632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749607c-6b3a-49f1-8797-40128fe32882 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.393812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 540/1084] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:03.399849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.400184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.400473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.404252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2221bc3-9241-4e70-9052-1eb9059b57cf None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:03.427621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:03.429002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:03.467985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:03.469029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.469526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.470008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.513744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 541/1085] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:06:03.514787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.554069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.555740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.603285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:03.649708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:03.655388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9477fd9-e2a5-4cd1-81b1-60e6559e661f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:03.700501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:03.701239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.701582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.701866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.711337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 545/1086] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:06:03.716679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.717650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.718032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:06:03.739633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:03.741278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:06:03.742224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:06:03.742224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:06:03.742224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:03.742224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 542/1087] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:03.748879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.845139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069382fd-5fc7-44a7-8843-c173b4c42678 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:03.867971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e25cb-abd3-4b16-ade7-0728da920856 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:03.881635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:03.888333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:03.889103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.892647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.892647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.902423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.904708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.911716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:03.912473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:03.912842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:03.913194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:03.913943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:03.914382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:03.916010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:06:03.922654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:03.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:06:03.925862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:06:03.925862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:06:03.925862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:06:03.925862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 546/1088] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:06:03.925862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.933857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 543/1089] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:03.935537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:03.959955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:04.116252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db38c10-546b-43cc-904d-08318fb1d2be None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.120244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c0459-b6f1-4f50-9305-7f5a57dc2a2d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.174501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.175143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.175480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.175710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.176862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:04.176862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.178128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.178253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.178892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.179923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.180248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.192615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 544/1090] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:04.197840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:04.198594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:04.202933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:04.204602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:06:04.205960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 547/1091] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:04.210414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.275855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:04.304358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:04.338012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d9cd9-95e2-4737-866b-57bb8d6572eb None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.345111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542d807-adf0-470c-a1ec-9fc33188abce None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.378237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.378809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.379083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.379364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.384649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.385289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.385573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.385852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.386544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:04.386887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.387156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.388478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.388819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.399693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 548/1092] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:06:04.401332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:04.402400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:04.408575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 545/1093] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:06:04.409181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.409258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.520919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75138b-9a4f-42cc-adce-8b9516ba5e84 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.525562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6864e-d372-4c84-9f75-dfdc34dca05e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.529890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:04.563174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.564286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.564733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.565192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] PATH_INFO: `/v3/users/d3a6591c8d6147bd91662eac7030e515` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.567009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] RBAC: Authorizing `identity:get_user(user_id=d3a6591c8d6147bd91662eac7030e515)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.569551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.570261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:04.576699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 549/1094] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:04 2022] GET /identity/v3/users/d3a6591c8d6147bd91662eac7030e515 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:04.578184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.581296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.582487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.582873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.583232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.585250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:04.585250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.585853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.599950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 546/1095] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:04.609590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.698320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf70ae7-b6a0-4a18-a151-15a1551e0b46 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.727672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c4380-a5e7-4430-be0f-0845c2757a8a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.738051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.738700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.738978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.739256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.740464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b8da3773674e46a0132a5181eef3d5)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.742039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.743767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 550/1096] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:04 2022] GET /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:04.750052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:04.766278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.766888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.767165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.767434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.768207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] RBAC: Adding query filter params (role.id=e6123810ab85413f8e3c86d465369675, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=3e5014eab4c7479f9ee493e31c91b3f4) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:04.768517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.769993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b9edcc-589d-4769-9885-339ed1682130 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:04.776361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 547/1097] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:04 2022] GET /identity/v3/role_assignments?user.id=3e5014eab4c7479f9ee493e31c91b3f4&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=e6123810ab85413f8e3c86d465369675 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:04.815917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:04.896532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c9280-8f88-49c8-9608-86dbe60e2cd7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:04.955750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db683d2e-ecf5-4a5a-8d1b-d554465d2a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:04.956859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db683d2e-ecf5-4a5a-8d1b-d554465d2a3a None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:04.957265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db683d2e-ecf5-4a5a-8d1b-d554465d2a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:04.957604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db683d2e-ecf5-4a5a-8d1b-d554465d2a3a None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5/users/d3a6591c8d6147bd91662eac7030e515/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:04.960873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db683d2e-ecf5-4a5a-8d1b-d554465d2a3a None admin] RBAC: Authorizing `identity:create_grant(project_id=c6b8da3773674e46a0132a5181eef3d5, user_id=d3a6591c8d6147bd91662eac7030e515, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:04.962439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db683d2e-ecf5-4a5a-8d1b-d554465d2a3a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:05.061258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 551/1098] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:06:04 2022] PUT /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5/users/d3a6591c8d6147bd91662eac7030e515/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:06:07.028614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf798d-6303-45d8-9b21-f0dad72324aa None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:07.029625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf798d-6303-45d8-9b21-f0dad72324aa None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:07.029625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf798d-6303-45d8-9b21-f0dad72324aa None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:07.030523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 548/1099] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:07.037812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:07.038334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:07.038676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:07.061886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:07.063061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:07.133753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 552/1100] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:07.400485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf798d-6303-45d8-9b21-f0dad72324aa None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:07.459812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:07.506119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf798d-6303-45d8-9b21-f0dad72324aa None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:07.548228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:07.549071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:07.549253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:07.549510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:07.564034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:07.568998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:07.570618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:07.571760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 549/1101] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:07.576863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:07.612426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:07.682258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e1485-e21d-4ca8-a750-75008f08692b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:07.718996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:07.719919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:07.720204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:07.720478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:07.721153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:07.721463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:07.722871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:07.731855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 553/1102] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:07.738514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:07.905234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84fc2b-1387-4b60-ae73-251eb699f7b4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:07.922489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:07.946942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:07.946942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:07.946942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:07.946942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] PATH_INFO: `/v3/users/d3a6591c8d6147bd91662eac7030e515` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:07.947424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] RBAC: Authorizing `identity:get_user(user_id=d3a6591c8d6147bd91662eac7030e515)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:07.947424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:07.947424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 550/1103] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:07 2022] GET /identity/v3/users/d3a6591c8d6147bd91662eac7030e515 => generated 285 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:07.952641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:08.071949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:08.079311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd356e-c23f-48ad-bd13-3796a13d2bcb None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:08.114219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:08.114826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:08.115103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:08.115371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:08.192525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:08.194000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:08.289383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:08.299453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 554/1104] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:08.314058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:08.410547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47719108-c5c4-4563-a3d7-43319c96379e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:08.445932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581e897-2cc6-4ace-9446-6e9f141b6d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:08.446574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e897-2cc6-4ace-9446-6e9f141b6d0c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:08.446858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e897-2cc6-4ace-9446-6e9f141b6d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:08.447251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581e897-2cc6-4ace-9446-6e9f141b6d0c None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:08.448428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581e897-2cc6-4ace-9446-6e9f141b6d0c None admin] RBAC: Authorizing `identity:get_project(project_id=c6b8da3773674e46a0132a5181eef3d5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:08.450425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581e897-2cc6-4ace-9446-6e9f141b6d0c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:08.452148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 551/1105] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:08 2022] GET /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:08.459588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:08.572270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7ffa6-cb34-40b7-b543-8d2594d02f87 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:08.625569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:08.626177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:08.626458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:08.626732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:08.627461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=c6b8da3773674e46a0132a5181eef3d5, user.id=d3a6591c8d6147bd91662eac7030e515) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:08.627768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:08.629237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d6ca-6fed-4d9d-a5a6-c6e2bdb42ddb None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:08.635234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 555/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:08 2022] GET /identity/v3/role_assignments?user.id=d3a6591c8d6147bd91662eac7030e515&scope.project.id=c6b8da3773674e46a0132a5181eef3d5&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:10.647116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8f729-e98d-4e2e-bbaa-c06453d297ea None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:10.647683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8f729-e98d-4e2e-bbaa-c06453d297ea None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:10.647683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8f729-e98d-4e2e-bbaa-c06453d297ea None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:10.649120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 552/1107] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:06:10.655278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:10.655746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:10.655833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:10.674685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:10.675907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:10.732764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:10.748683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 556/1108] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:11.024872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8f729-e98d-4e2e-bbaa-c06453d297ea None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:11.096606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:11.149824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8f729-e98d-4e2e-bbaa-c06453d297ea None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:11.185167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e7443-7e68-4ae8-a048-a616d590b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:11.185813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e7443-7e68-4ae8-a048-a616d590b1bc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:11.186129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e7443-7e68-4ae8-a048-a616d590b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:11.186432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e7443-7e68-4ae8-a048-a616d590b1bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:11.187594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e7443-7e68-4ae8-a048-a616d590b1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:11.189088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e7443-7e68-4ae8-a048-a616d590b1bc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:11.190687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 553/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:06:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:11.196617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:11.299494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c0868-d457-4b7e-8e6a-d9c08ba3ac04 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:11.337024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954171ca-b82a-45ea-bd06-fae26deb90b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:11.337681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954171ca-b82a-45ea-bd06-fae26deb90b1 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:11.337976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954171ca-b82a-45ea-bd06-fae26deb90b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:11.338231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954171ca-b82a-45ea-bd06-fae26deb90b1 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:11.338907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954171ca-b82a-45ea-bd06-fae26deb90b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:11.340460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954171ca-b82a-45ea-bd06-fae26deb90b1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:11.511654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 557/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:06:11 2022] POST /identity/v3/users => generated 286 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:06:13.551737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141c1a-d81f-4cc6-a3ac-7e37a5a0c556 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:13.552360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141c1a-d81f-4cc6-a3ac-7e37a5a0c556 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:13.552618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141c1a-d81f-4cc6-a3ac-7e37a5a0c556 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:13.554469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 554/1111] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:06:13.564555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581955b6-c93b-433d-8690-7ba894930472 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:13.565072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581955b6-c93b-433d-8690-7ba894930472 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:13.565536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581955b6-c93b-433d-8690-7ba894930472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:13.599196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-581955b6-c93b-433d-8690-7ba894930472 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:13.601650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-581955b6-c93b-433d-8690-7ba894930472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:13.672198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:13.713206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 558/1112] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:13.984549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9141c1a-d81f-4cc6-a3ac-7e37a5a0c556 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:14.124718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9141c1a-d81f-4cc6-a3ac-7e37a5a0c556 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:14.157026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:14.175878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:14.176648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:14.176928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:14.177248 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] PATH_INFO: `/v3/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:14.178716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] RBAC: Authorizing `identity:get_role(role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:14.180183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:14.183229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 555/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:13 2022] GET /identity/v3/roles/c2a3834a436b40deabcb87f412ca5322 => generated 223 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:14.190046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581955b6-c93b-433d-8690-7ba894930472 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:14.313331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581955b6-c93b-433d-8690-7ba894930472 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:14.351379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:14.352064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:14.352347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:14.352618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] PATH_INFO: `/v3/users/0abe05ddc7e94419a0451bbc0cef58a8` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:14.367712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] RBAC: Authorizing `identity:get_user(user_id=0abe05ddc7e94419a0451bbc0cef58a8)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:14.369209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:14.371471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 559/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:14 2022] GET /identity/v3/users/0abe05ddc7e94419a0451bbc0cef58a8 => generated 286 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:14.377769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:14.487177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177a344-fbb5-4cab-aee4-7db346ca9982 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:14.535686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:14.536331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:14.536609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:14.536885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:14.594713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:14.607396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:14.608899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:14.714869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 556/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:14.721043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:14.755362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:14.830736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51372b87-c15a-46e5-b1f3-f8a6d48caf4f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:14.867318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f08308-e818-4e95-a39e-c8abce1dd617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:14.867872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f08308-e818-4e95-a39e-c8abce1dd617 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:14.868179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f08308-e818-4e95-a39e-c8abce1dd617 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:14.868451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f08308-e818-4e95-a39e-c8abce1dd617 None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:14.869570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f08308-e818-4e95-a39e-c8abce1dd617 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b8da3773674e46a0132a5181eef3d5)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:14.871048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f08308-e818-4e95-a39e-c8abce1dd617 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:14.872734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 560/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:14 2022] GET /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:14.879241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:14.974529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:14.988621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69a711-c266-4a7d-a9b4-3833bf213cf4 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:15.035754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:15.036579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:15.036857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:15.037132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:15.037873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] RBAC: Adding query filter params (role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=c6b8da3773674e46a0132a5181eef3d5, user.id=0abe05ddc7e94419a0451bbc0cef58a8) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:15.038183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:15.039621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad0370-e233-4915-b494-38b078708cb5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:15.046488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 557/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:14 2022] GET /identity/v3/role_assignments?user.id=0abe05ddc7e94419a0451bbc0cef58a8&scope.project.id=c6b8da3773674e46a0132a5181eef3d5&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:17.012727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc95eb-9f32-49ee-9a68-6b4f180c1053 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:17.013458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc95eb-9f32-49ee-9a68-6b4f180c1053 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:17.013458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc95eb-9f32-49ee-9a68-6b4f180c1053 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:17.014866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 561/1118] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:17.021404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:17.021925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:17.022035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:17.042983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:17.044231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:17.128908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 558/1119] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:17.409189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc95eb-9f32-49ee-9a68-6b4f180c1053 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:17.524416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:17.529060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc95eb-9f32-49ee-9a68-6b4f180c1053 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:17.568859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:17.569608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:17.569994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:17.570352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] PATH_INFO: `/v3/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:17.571846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] RBAC: Authorizing `identity:get_role(role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:17.573884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:17.575284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 562/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:17 2022] GET /identity/v3/roles/c2a3834a436b40deabcb87f412ca5322 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:17.581322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:17.662257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:17.680063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffe86a-7f98-4c83-9c87-a9493ba44efc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:17.718472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:17.719204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:17.719526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:17.719872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] PATH_INFO: `/v3/users/0abe05ddc7e94419a0451bbc0cef58a8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:17.721178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] RBAC: Authorizing `identity:get_user(user_id=0abe05ddc7e94419a0451bbc0cef58a8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:17.722731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:17.724989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 559/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:17 2022] GET /identity/v3/users/0abe05ddc7e94419a0451bbc0cef58a8 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:17.731172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:17.849411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8279e-f122-4d61-ac36-c4bc55071cc1 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:17.895132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689e377-0b30-49e8-9189-8da2b5485b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:17.895773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689e377-0b30-49e8-9189-8da2b5485b54 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:17.896096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689e377-0b30-49e8-9189-8da2b5485b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:17.896348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689e377-0b30-49e8-9189-8da2b5485b54 None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:17.897534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689e377-0b30-49e8-9189-8da2b5485b54 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b8da3773674e46a0132a5181eef3d5)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:17.899002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689e377-0b30-49e8-9189-8da2b5485b54 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:17.900771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 563/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:17 2022] GET /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:17.907331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:18.017762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425641fd-72d1-47a2-8e1f-71c2df12539d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:18.056624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a7e4a-5d7e-4300-a09d-12284c4ee3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:18.057818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7e4a-5d7e-4300-a09d-12284c4ee3e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:18.058136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7e4a-5d7e-4300-a09d-12284c4ee3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:18.058422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a7e4a-5d7e-4300-a09d-12284c4ee3e3 None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5/users/0abe05ddc7e94419a0451bbc0cef58a8/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:18.060813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a7e4a-5d7e-4300-a09d-12284c4ee3e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6b8da3773674e46a0132a5181eef3d5, user_id=0abe05ddc7e94419a0451bbc0cef58a8, role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:18.062270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a7e4a-5d7e-4300-a09d-12284c4ee3e3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:18.138896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 560/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:06:17 2022] PUT /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5/users/0abe05ddc7e94419a0451bbc0cef58a8/roles/c2a3834a436b40deabcb87f412ca5322 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:06:20.182854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5132-a8c5-4477-9708-f5761f265d68 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:20.183983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5132-a8c5-4477-9708-f5761f265d68 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:20.183983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c5132-a8c5-4477-9708-f5761f265d68 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:20.185947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 564/1124] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:20.193958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:20.194786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:20.194786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:20.222504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:20.224058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:20.227988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:20.325330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 561/1125] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:20.617624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c5132-a8c5-4477-9708-f5761f265d68 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:20.657856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:20.725928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c5132-a8c5-4477-9708-f5761f265d68 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:20.764726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:20.765473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:20.765757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:20.766033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] PATH_INFO: `/v3/roles/c2a3834a436b40deabcb87f412ca5322` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:20.767477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] RBAC: Authorizing `identity:get_role(role_id=c2a3834a436b40deabcb87f412ca5322)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:20.768898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:20.770262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 565/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:20 2022] GET /identity/v3/roles/c2a3834a436b40deabcb87f412ca5322 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:20.776366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:20.884703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d1152-0a51-4974-90c6-0b7c8e100bc9 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:20.920082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:20.920752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:20.921033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:20.921326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] PATH_INFO: `/v3/users/0abe05ddc7e94419a0451bbc0cef58a8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:20.922517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] RBAC: Authorizing `identity:get_user(user_id=0abe05ddc7e94419a0451bbc0cef58a8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:20.923988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:20.925620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 562/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:20 2022] GET /identity/v3/users/0abe05ddc7e94419a0451bbc0cef58a8 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:20.931652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:21.045781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414253ba-5411-4c2f-880c-68d5e996eb3a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:21.048615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:21.082824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:21.083450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:21.083727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:21.084075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:21.158872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:21.160283 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:21.269146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 566/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:21.275437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:21.286954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:21.382890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21241d7c-1d0a-4cad-98d6-81ccc970161e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:21.421176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01453fbe-5829-438f-9ed0-e238c6bd3c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:21.421786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01453fbe-5829-438f-9ed0-e238c6bd3c39 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:21.422126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01453fbe-5829-438f-9ed0-e238c6bd3c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:21.422430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01453fbe-5829-438f-9ed0-e238c6bd3c39 None admin] PATH_INFO: `/v3/projects/c6b8da3773674e46a0132a5181eef3d5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:21.423680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01453fbe-5829-438f-9ed0-e238c6bd3c39 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b8da3773674e46a0132a5181eef3d5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:21.425468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01453fbe-5829-438f-9ed0-e238c6bd3c39 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:21.427620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 563/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:21 2022] GET /identity/v3/projects/c6b8da3773674e46a0132a5181eef3d5 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:21.434235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:21.437724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:21.548190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205735a9-6891-4a0d-a06f-ff46bf008455 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:21.587353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:21.588753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:21.589183 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:21.589552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:21.590402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] RBAC: Adding query filter params (role.id=c2a3834a436b40deabcb87f412ca5322, scope.project.id=c6b8da3773674e46a0132a5181eef3d5, user.id=0abe05ddc7e94419a0451bbc0cef58a8) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:21.590810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:21.592938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66daeb8-4218-4575-9c3d-b33410461788 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:21.601134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 567/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:21 2022] GET /identity/v3/role_assignments?user.id=0abe05ddc7e94419a0451bbc0cef58a8&scope.project.id=c6b8da3773674e46a0132a5181eef3d5&role.id=c2a3834a436b40deabcb87f412ca5322 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:23.593031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f939a4-4803-4cb8-a4e2-5f3b8ab98917 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:23.593587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f939a4-4803-4cb8-a4e2-5f3b8ab98917 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:23.593587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f939a4-4803-4cb8-a4e2-5f3b8ab98917 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:23.594945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 564/1131] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:23.601045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:23.601590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:23.601590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:23.622010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:23.623235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:23.688433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:23.694002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 568/1132] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:23.964943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f939a4-4803-4cb8-a4e2-5f3b8ab98917 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:24.044609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:24.069799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f939a4-4803-4cb8-a4e2-5f3b8ab98917 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:24.122526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c085f5-f2f9-44c7-848a-6b87d5bd0e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:24.123208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c085f5-f2f9-44c7-848a-6b87d5bd0e44 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:24.123492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c085f5-f2f9-44c7-848a-6b87d5bd0e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:24.123778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c085f5-f2f9-44c7-848a-6b87d5bd0e44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:24.125182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c085f5-f2f9-44c7-848a-6b87d5bd0e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:24.126715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c085f5-f2f9-44c7-848a-6b87d5bd0e44 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:24.128635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 565/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:06:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:24.135211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:24.249738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7286-6bb8-4121-8ea7-261948b337ff None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:24.288337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27beea52-7de9-449a-b908-cd7aea57b4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:24.288969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beea52-7de9-449a-b908-cd7aea57b4f1 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:24.289250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beea52-7de9-449a-b908-cd7aea57b4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:24.289526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beea52-7de9-449a-b908-cd7aea57b4f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:24.290197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27beea52-7de9-449a-b908-cd7aea57b4f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:24.291694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27beea52-7de9-449a-b908-cd7aea57b4f1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:24.402279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 569/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:06:24 2022] POST /identity/v3/projects => generated 311 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:06:26.394793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0ca8-9d28-48ca-be90-65c5f2ea77fc None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:26.395303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0ca8-9d28-48ca-be90-65c5f2ea77fc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:26.395303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0ca8-9d28-48ca-be90-65c5f2ea77fc None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:26.396657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 566/1135] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:26.402511 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:26.402998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:26.403115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:26.423038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:26.424284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:26.474958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:26.498697 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 570/1136] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:26.774336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0ca8-9d28-48ca-be90-65c5f2ea77fc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:26.878413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0ca8-9d28-48ca-be90-65c5f2ea77fc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:26.913529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1178112-ce40-4e9f-85c5-a667539a1019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:26.914173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178112-ce40-4e9f-85c5-a667539a1019 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:26.914456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178112-ce40-4e9f-85c5-a667539a1019 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:26.914730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178112-ce40-4e9f-85c5-a667539a1019 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:26.916502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1178112-ce40-4e9f-85c5-a667539a1019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:26.917946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1178112-ce40-4e9f-85c5-a667539a1019 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:26.919551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 567/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:06:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:26.925637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:27.034134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1094c3-0701-4ef0-bf0c-c3fee1311076 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:27.070495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ec0e7-5a13-4d73-b5bf-4f8d85021700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:27.071134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ec0e7-5a13-4d73-b5bf-4f8d85021700 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:27.071417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ec0e7-5a13-4d73-b5bf-4f8d85021700 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:27.071693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ec0e7-5a13-4d73-b5bf-4f8d85021700 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:27.072438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ec0e7-5a13-4d73-b5bf-4f8d85021700 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:27.073895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ec0e7-5a13-4d73-b5bf-4f8d85021700 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:27.187032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 571/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:06:26 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:06:29.170799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8146-5403-43a3-96f1-3b2723b485cc None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:29.171367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8146-5403-43a3-96f1-3b2723b485cc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:29.171463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8146-5403-43a3-96f1-3b2723b485cc None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:29.172899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 568/1139] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:29.178935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:29.179484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:29.179484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:29.199318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:29.200567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:29.229994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:29.270901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 572/1140] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:29.542304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a8146-5403-43a3-96f1-3b2723b485cc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:29.558921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:29.655111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a8146-5403-43a3-96f1-3b2723b485cc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:29.693152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:29.693887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:29.694200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:29.694555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:29.706181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:29.711695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:29.712938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:29.713973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:29.716052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 569/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:29.719128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:29.851075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a9ea8-fb31-4dc2-8487-4823f37abd44 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:29.902711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:29.903460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:29.903972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:29.904420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:29.905714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:29.906248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:29.908769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:29.920429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 573/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:29.934783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:30.018470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:30.048927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4187e-1553-444d-a053-57eb276c1d61 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:30.093781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:30.094452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:30.094812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:30.095108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] PATH_INFO: `/v3/users/2fc9ce19cd0447c188dafe541cfb0a9a` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:30.117185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] RBAC: Authorizing `identity:get_user(user_id=2fc9ce19cd0447c188dafe541cfb0a9a)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:30.119356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:30.121157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 570/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:29 2022] GET /identity/v3/users/2fc9ce19cd0447c188dafe541cfb0a9a => generated 286 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:30.128040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:30.169550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:30.256564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d6aee-798e-46c1-b94e-6c17280b09c0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:30.290983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:30.291638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:30.291946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:30.292229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:30.366572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:30.368142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:30.399102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:30.480322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 574/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:30.486536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:30.602591 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e4989-18aa-4d06-bd64-f37e1e4acdfb None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:30.639732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:30.646553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297e50-05c6-464e-821f-407ac87d1105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:30.647227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297e50-05c6-464e-821f-407ac87d1105 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:30.647515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297e50-05c6-464e-821f-407ac87d1105 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:30.647815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297e50-05c6-464e-821f-407ac87d1105 None admin] PATH_INFO: `/v3/projects/c7ce523ff76c4707884d070e4e0b2147` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:30.649036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297e50-05c6-464e-821f-407ac87d1105 None admin] RBAC: Authorizing `identity:get_project(project_id=c7ce523ff76c4707884d070e4e0b2147)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:30.651196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297e50-05c6-464e-821f-407ac87d1105 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:30.652948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 571/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:30 2022] GET /identity/v3/projects/c7ce523ff76c4707884d070e4e0b2147 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:30.659933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:30.771724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb748672-debc-45f2-9a98-b0f8cb57f408 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:30.800661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:30.808491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:30.814246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:30.814246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:30.814246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:30.814246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=c7ce523ff76c4707884d070e4e0b2147, user.id=2fc9ce19cd0447c188dafe541cfb0a9a) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:30.814246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:30.814246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f76556-aff3-4f87-a49a-d9061b3f4145 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:30.817765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 575/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:30 2022] GET /identity/v3/role_assignments?user.id=2fc9ce19cd0447c188dafe541cfb0a9a&scope.project.id=c7ce523ff76c4707884d070e4e0b2147&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:32.714478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13e07-6777-47bc-8e4a-62cccd431f4f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:32.715103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13e07-6777-47bc-8e4a-62cccd431f4f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:32.715103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b13e07-6777-47bc-8e4a-62cccd431f4f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:32.716534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 572/1147] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:32.722628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:32.723185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:32.723185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:32.742754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:32.744047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:32.822922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 576/1148] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:33.088045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b13e07-6777-47bc-8e4a-62cccd431f4f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:33.190054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b13e07-6777-47bc-8e4a-62cccd431f4f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:33.225826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:33.226460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:33.226744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:33.227021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:33.242427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:33.248074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:33.249814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:33.250992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 573/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:33.255476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:33.361449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b007607-e309-4c43-99d7-58526cd0bffd None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:33.385323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:33.397031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:33.397686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:33.398004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:33.403095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:33.403095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:33.403095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:33.403095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:33.410144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 577/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:33.416613 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:33.478218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:33.521105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c513e0-105a-404c-9118-565ae2d3797e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:33.560091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:33.560727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:33.561007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:33.561282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] PATH_INFO: `/v3/users/2fc9ce19cd0447c188dafe541cfb0a9a` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:33.562418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] RBAC: Authorizing `identity:get_user(user_id=2fc9ce19cd0447c188dafe541cfb0a9a)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:33.563864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:33.565518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 574/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:33 2022] GET /identity/v3/users/2fc9ce19cd0447c188dafe541cfb0a9a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:33.571421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:33.674912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964cae29-1e6b-4406-b878-3eff40cb0f16 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:33.708242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d0238-6fa4-4de1-a4c1-67f55f3624bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:33.708878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d0238-6fa4-4de1-a4c1-67f55f3624bd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:33.709158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d0238-6fa4-4de1-a4c1-67f55f3624bd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:33.712251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d0238-6fa4-4de1-a4c1-67f55f3624bd None admin] PATH_INFO: `/v3/projects/c7ce523ff76c4707884d070e4e0b2147` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:33.712251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d0238-6fa4-4de1-a4c1-67f55f3624bd None admin] RBAC: Authorizing `identity:get_project(project_id=c7ce523ff76c4707884d070e4e0b2147)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:33.712554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d0238-6fa4-4de1-a4c1-67f55f3624bd None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:33.714223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 578/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:33 2022] GET /identity/v3/projects/c7ce523ff76c4707884d070e4e0b2147 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:33.720453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:33.827714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8405bd0-b7d1-4685-b467-278f6d781e28 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:33.854731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:33.863547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d996b-c593-491a-bb63-25472a18f623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:33.864256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d996b-c593-491a-bb63-25472a18f623 None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:33.866762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d996b-c593-491a-bb63-25472a18f623 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:33.866762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d996b-c593-491a-bb63-25472a18f623 None admin] PATH_INFO: `/v3/projects/c7ce523ff76c4707884d070e4e0b2147/users/2fc9ce19cd0447c188dafe541cfb0a9a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:33.866762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d996b-c593-491a-bb63-25472a18f623 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7ce523ff76c4707884d070e4e0b2147, user_id=2fc9ce19cd0447c188dafe541cfb0a9a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:33.868258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d996b-c593-491a-bb63-25472a18f623 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:34.049541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 575/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:06:33 2022] PUT /identity/v3/projects/c7ce523ff76c4707884d070e4e0b2147/users/2fc9ce19cd0447c188dafe541cfb0a9a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:06:35.964407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d274-6674-4a7f-84bd-f71fcd361403 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:35.965008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d274-6674-4a7f-84bd-f71fcd361403 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:35.965008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d274-6674-4a7f-84bd-f71fcd361403 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:35.966358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 579/1154] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:35.972186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:35.972675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:35.972824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:35.992388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:35.993595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:36.071740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 576/1155] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:36.340897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2d274-6674-4a7f-84bd-f71fcd361403 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:36.406643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:36.455516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2d274-6674-4a7f-84bd-f71fcd361403 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:36.492592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:36.493256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:36.493534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:36.493806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:36.504489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:36.509885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:36.511087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:06:36.512233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 580/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:36.517113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:36.620570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5662cc-db17-4d0b-a084-80e64144da1a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:36.638081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:36.655745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:36.656389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:36.656672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:36.656941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:36.657608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:36.657909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:36.659282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:36.668217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 577/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:36.674124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:36.782361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4106ef-5ea2-483e-9b48-c373800600dd None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:36.796428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:36.819446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:36.820148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:36.820394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:36.820667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] PATH_INFO: `/v3/users/2fc9ce19cd0447c188dafe541cfb0a9a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:36.822417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] RBAC: Authorizing `identity:get_user(user_id=2fc9ce19cd0447c188dafe541cfb0a9a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:36.823864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:36.825851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 581/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:36 2022] GET /identity/v3/users/2fc9ce19cd0447c188dafe541cfb0a9a => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:36.831977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:36.941406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806f439-c762-476a-bea2-c9fae72d5457 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:36.976189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:36.976859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:36.977595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:36.977864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:37.022768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:37.044205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:37.045687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:37.146128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 578/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:37.152129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:37.254819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dbc61-83b3-4dcd-937e-b74e58701b22 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:37.295652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b873549-5a21-46e7-92f7-3592f00281e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:37.296319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873549-5a21-46e7-92f7-3592f00281e1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:37.296601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873549-5a21-46e7-92f7-3592f00281e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:37.296875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873549-5a21-46e7-92f7-3592f00281e1 None admin] PATH_INFO: `/v3/projects/c7ce523ff76c4707884d070e4e0b2147` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:37.298048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b873549-5a21-46e7-92f7-3592f00281e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c7ce523ff76c4707884d070e4e0b2147)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:37.299498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b873549-5a21-46e7-92f7-3592f00281e1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:37.301197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 582/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:37 2022] GET /identity/v3/projects/c7ce523ff76c4707884d070e4e0b2147 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:37.307958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:37.385525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:37.408180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b249-deac-43cf-9d13-d89310b42b5d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:37.448967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:37.449620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:37.449901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:37.450178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:37.450906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=c7ce523ff76c4707884d070e4e0b2147, user.id=2fc9ce19cd0447c188dafe541cfb0a9a) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:37.451218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:37.452681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17456fa5-3188-43b6-9f1e-b14001297edc None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:37.458893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 579/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:37 2022] GET /identity/v3/role_assignments?user.id=2fc9ce19cd0447c188dafe541cfb0a9a&scope.project.id=c7ce523ff76c4707884d070e4e0b2147&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:39.389201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16483f4f-57df-4f59-baa3-1bc8cea2f8b9 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:39.389708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16483f4f-57df-4f59-baa3-1bc8cea2f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:39.389708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16483f4f-57df-4f59-baa3-1bc8cea2f8b9 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:39.391006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 583/1162] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:39.396180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:39.396567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:39.396731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:39.415130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:39.416319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:39.482170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 580/1163] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:39.745567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16483f4f-57df-4f59-baa3-1bc8cea2f8b9 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:39.795429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:39.845042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16483f4f-57df-4f59-baa3-1bc8cea2f8b9 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:39.877844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:39.878451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:39.878735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:39.879012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:39.944387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:39.945845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:39.993668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:40.047933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 584/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:40.053413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:40.122711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:40.152629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c341c24-3d49-472a-ad8c-eca3a8a7b848 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:40.185761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:40.186387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:40.186664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:40.186935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:40.193475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:40.194909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-8357ac0f-3203-4f16-bdec-9ce4b11344b3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:40.206585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:40.207802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:40.208976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 581/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:06:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:40.214385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:40.310444 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037262e-f25a-4edb-88a9-7921f2e1d755 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:40.343699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:40.344366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:40.344647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:40.344923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:40.345559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:40.345863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:40.347274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ca9f8-5d02-482a-b4ec-26ca4c12a928 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:40.354574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 585/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Mon Aug 29 21:06:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:42.244972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d4e0-576a-49eb-a720-5b4f23fa3365 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:42.245555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d4e0-576a-49eb-a720-5b4f23fa3365 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:42.245555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0d4e0-576a-49eb-a720-5b4f23fa3365 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:42.246864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 582/1167] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:42.252005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c629271-e1b4-4d76-91dc-47de6e681ea3 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:42.252420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c629271-e1b4-4d76-91dc-47de6e681ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:42.252563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c629271-e1b4-4d76-91dc-47de6e681ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:42.270478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2c629271-e1b4-4d76-91dc-47de6e681ea3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:42.271653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2c629271-e1b4-4d76-91dc-47de6e681ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:42.278856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:42.337522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 586/1168] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:42.609733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0d4e0-576a-49eb-a720-5b4f23fa3365 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:42.709839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0d4e0-576a-49eb-a720-5b4f23fa3365 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:42.739717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:42.745171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a071fcb-80bc-442f-9266-cacd6bb62c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:42.745807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a071fcb-80bc-442f-9266-cacd6bb62c7d None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:42.746079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a071fcb-80bc-442f-9266-cacd6bb62c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:42.746348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a071fcb-80bc-442f-9266-cacd6bb62c7d None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:42.747021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a071fcb-80bc-442f-9266-cacd6bb62c7d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:42.748420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a071fcb-80bc-442f-9266-cacd6bb62c7d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:42.855116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 583/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Mon Aug 29 21:06:42 2022] POST /identity/v3/domains => generated 267 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:06:44.766685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3f96f-b5cd-4ac1-af9e-f15b87b42e22 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:44.767216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3f96f-b5cd-4ac1-af9e-f15b87b42e22 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:44.767216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3f96f-b5cd-4ac1-af9e-f15b87b42e22 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:44.768796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 587/1170] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:44.774556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:44.774963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:44.775092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:44.793766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:44.794957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:44.860987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 584/1171] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:45.127688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3f96f-b5cd-4ac1-af9e-f15b87b42e22 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:45.230566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3f96f-b5cd-4ac1-af9e-f15b87b42e22 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:45.272478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec44e34-cf7c-451c-a589-acc42647106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:45.273116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec44e34-cf7c-451c-a589-acc42647106c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:45.273405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec44e34-cf7c-451c-a589-acc42647106c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:45.273718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec44e34-cf7c-451c-a589-acc42647106c None admin] PATH_INFO: `/v3/domains/fdd26099e7c74b6aaea8e9fc14b6c65d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:45.284909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec44e34-cf7c-451c-a589-acc42647106c None admin] RBAC: Authorizing `identity:get_domain(domain_id=fdd26099e7c74b6aaea8e9fc14b6c65d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:45.286380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec44e34-cf7c-451c-a589-acc42647106c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:45.288122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 588/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Mon Aug 29 21:06:45 2022] GET /identity/v3/domains/fdd26099e7c74b6aaea8e9fc14b6c65d => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:45.296306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:45.406859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffcc36-e032-4d9a-b144-12c8b58c6b70 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:45.430275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:45.442487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a62ac-4d2d-4baa-8daf-04b2449844f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:45.443179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a62ac-4d2d-4baa-8daf-04b2449844f2 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:45.443460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a62ac-4d2d-4baa-8daf-04b2449844f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:45.443735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a62ac-4d2d-4baa-8daf-04b2449844f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:45.444428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a62ac-4d2d-4baa-8daf-04b2449844f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:45.446486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a62ac-4d2d-4baa-8daf-04b2449844f2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:45.552707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 585/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:06:45 2022] POST /identity/v3/projects => generated 361 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:06:47.528326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71606469-2ace-4c80-8247-6fc50a77fcc9 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:47.528845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71606469-2ace-4c80-8247-6fc50a77fcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:47.528845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71606469-2ace-4c80-8247-6fc50a77fcc9 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:47.530149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 589/1174] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:47.535464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:47.535856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:47.536055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:47.554567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:47.555808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:47.621459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 586/1175] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:47.894893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71606469-2ace-4c80-8247-6fc50a77fcc9 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:47.897831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:48.000152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71606469-2ace-4c80-8247-6fc50a77fcc9 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:48.034268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305bd22-b149-4699-990e-402351823394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:48.034901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305bd22-b149-4699-990e-402351823394 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:48.035176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305bd22-b149-4699-990e-402351823394 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:48.035447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305bd22-b149-4699-990e-402351823394 None admin] PATH_INFO: `/v3/domains/fdd26099e7c74b6aaea8e9fc14b6c65d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:48.036679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305bd22-b149-4699-990e-402351823394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fdd26099e7c74b6aaea8e9fc14b6c65d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:48.038115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305bd22-b149-4699-990e-402351823394 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:48.039684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 590/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Mon Aug 29 21:06:47 2022] GET /identity/v3/domains/fdd26099e7c74b6aaea8e9fc14b6c65d => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:48.045823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:48.145142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f088d-c7ae-4338-97a1-2e09243adbcf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:48.147382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:48.178070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa75f3-152c-4cfc-b032-15124dd6197d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:48.179000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa75f3-152c-4cfc-b032-15124dd6197d None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:48.179361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa75f3-152c-4cfc-b032-15124dd6197d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:48.179842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa75f3-152c-4cfc-b032-15124dd6197d None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:48.181469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa75f3-152c-4cfc-b032-15124dd6197d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:48.182955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa75f3-152c-4cfc-b032-15124dd6197d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:48.328315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 587/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:06:48 2022] POST /identity/v3/users => generated 311 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:06:50.271695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021a2c-0a61-48a6-9ca5-cf99202c9c05 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:50.272277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021a2c-0a61-48a6-9ca5-cf99202c9c05 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:50.272277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021a2c-0a61-48a6-9ca5-cf99202c9c05 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:50.273617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 591/1178] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:50.279865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:50.280391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:50.280535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:50.298541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:50.299735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:50.367601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 588/1179] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:50.638986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63021a2c-0a61-48a6-9ca5-cf99202c9c05 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:50.713636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:50.752090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63021a2c-0a61-48a6-9ca5-cf99202c9c05 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:50.786510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:50.789278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:50.789278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:50.789278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:50.798943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:50.803791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:50.804970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:06:50.806566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 592/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:50.811653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:50.903698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:50.913164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab48607-57c7-4fbe-8488-84baac81333e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:50.954012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:50.954637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:50.954919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:50.955186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:50.955852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:50.956192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:50.957576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:50.965918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 589/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:50.972892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:51.077000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8290bd3-a984-43b3-b537-c090423bc711 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:51.107406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:51.113024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:51.113754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:51.114042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:51.114322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] PATH_INFO: `/v3/users/c9a0c30e88ea490cb228e9d27a6b78be` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:51.130014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a0c30e88ea490cb228e9d27a6b78be)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:51.131843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:51.132732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 593/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:50 2022] GET /identity/v3/users/c9a0c30e88ea490cb228e9d27a6b78be => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:51.138815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:51.240226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178fb1e-4543-491f-b5e2-2312fd8d6406 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:51.273016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:51.273631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:51.274400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:51.274706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:51.296911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:51.348617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:51.350080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:51.447242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 590/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:51.453324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:51.555821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317577a0-108d-4742-a202-29fe301e2c87 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:51.593046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446fc5a-15b7-43f0-93e9-1b81caa284b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:51.593675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446fc5a-15b7-43f0-93e9-1b81caa284b8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:51.593953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446fc5a-15b7-43f0-93e9-1b81caa284b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:51.594280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446fc5a-15b7-43f0-93e9-1b81caa284b8 None admin] PATH_INFO: `/v3/projects/68a6fc3b0be04272bafca71183f4f351` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:51.595397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446fc5a-15b7-43f0-93e9-1b81caa284b8 None admin] RBAC: Authorizing `identity:get_project(project_id=68a6fc3b0be04272bafca71183f4f351)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:51.596862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446fc5a-15b7-43f0-93e9-1b81caa284b8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:51.598786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 594/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:51 2022] GET /identity/v3/projects/68a6fc3b0be04272bafca71183f4f351 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:51.605352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:51.659439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:51.707509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5c0e6-d38a-433e-8b99-281d8d0daab6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:51.742422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:51.743055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:51.743325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:51.743609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:51.744372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=68a6fc3b0be04272bafca71183f4f351, user.id=c9a0c30e88ea490cb228e9d27a6b78be) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:51.744684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:51.746128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f74f2-b105-4a22-b28e-362bea47c1a5 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:51.751749 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 591/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:51 2022] GET /identity/v3/role_assignments?user.id=c9a0c30e88ea490cb228e9d27a6b78be&scope.project.id=68a6fc3b0be04272bafca71183f4f351&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:53.622035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4a455-458d-476e-a9d1-9d1ee6771861 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:53.622585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4a455-458d-476e-a9d1-9d1ee6771861 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:53.622791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4a455-458d-476e-a9d1-9d1ee6771861 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:53.624187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 595/1186] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:53.629874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:53.630267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:53.630501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:53.649848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:53.651086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:53.720701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 592/1187] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:53.990665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4a455-458d-476e-a9d1-9d1ee6771861 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:54.047444 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:54.090824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4a455-458d-476e-a9d1-9d1ee6771861 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:54.127028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:54.127591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:54.127856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:54.128157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:54.138506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:54.143343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:54.145000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:06:54.146240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 596/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:06:54.150653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:54.200808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:54.253375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e2e2-cf5e-43e2-acfe-055241f8f3f6 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:54.287823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:54.288464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:54.288745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:54.289019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:54.290301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:54.290605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:54.292006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:54.300132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 593/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:54.306102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:54.406706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a8cc-26d3-4434-8946-205973b170ed None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:54.423621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:54.441265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:54.441911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:54.442194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:54.442465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] PATH_INFO: `/v3/users/c9a0c30e88ea490cb228e9d27a6b78be` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:54.443598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a0c30e88ea490cb228e9d27a6b78be)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:54.445163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:54.446773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 597/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:54 2022] GET /identity/v3/users/c9a0c30e88ea490cb228e9d27a6b78be => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:54.452889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:54.554919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d6152-8e72-4cfd-8a72-a99f97a31d25 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:54.563819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:54.588761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed77dff-4662-4136-85b7-8edf4ef293c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:54.589402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77dff-4662-4136-85b7-8edf4ef293c2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:54.589679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77dff-4662-4136-85b7-8edf4ef293c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:54.589954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed77dff-4662-4136-85b7-8edf4ef293c2 None admin] PATH_INFO: `/v3/projects/68a6fc3b0be04272bafca71183f4f351` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:54.591106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77dff-4662-4136-85b7-8edf4ef293c2 None admin] RBAC: Authorizing `identity:get_project(project_id=68a6fc3b0be04272bafca71183f4f351)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:54.592581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed77dff-4662-4136-85b7-8edf4ef293c2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:54.594231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 594/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:54 2022] GET /identity/v3/projects/68a6fc3b0be04272bafca71183f4f351 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:54.600671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:54.698968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e21e1-ca08-43e5-97e7-27b8771f25f9 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:54.732876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0fbe72-bdbb-451e-9596-03358b2fe984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:54.733542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbe72-bdbb-451e-9596-03358b2fe984 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:54.733823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbe72-bdbb-451e-9596-03358b2fe984 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:54.734093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0fbe72-bdbb-451e-9596-03358b2fe984 None admin] PATH_INFO: `/v3/projects/68a6fc3b0be04272bafca71183f4f351/users/c9a0c30e88ea490cb228e9d27a6b78be/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:54.736054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbe72-bdbb-451e-9596-03358b2fe984 None admin] RBAC: Authorizing `identity:create_grant(project_id=68a6fc3b0be04272bafca71183f4f351, user_id=c9a0c30e88ea490cb228e9d27a6b78be, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:54.737497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0fbe72-bdbb-451e-9596-03358b2fe984 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:54.864742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 598/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:06:54 2022] PUT /identity/v3/projects/68a6fc3b0be04272bafca71183f4f351/users/c9a0c30e88ea490cb228e9d27a6b78be/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:06:56.750076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118f256-2cea-4bfa-974d-8aab2a1b657d None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:56.750616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118f256-2cea-4bfa-974d-8aab2a1b657d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:56.750616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118f256-2cea-4bfa-974d-8aab2a1b657d None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:56.752006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 595/1193] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:06:56.759153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:56.759632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:56.759722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:56.777957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:06:56.779138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:06:56.818765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:56.844862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 599/1194] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:06:57.113406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118f256-2cea-4bfa-974d-8aab2a1b657d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:57.220453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118f256-2cea-4bfa-974d-8aab2a1b657d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:57.258452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:57.259082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:57.259363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:57.259628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:57.269537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:06:57.274832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:06:57.275966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:06:57.277112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 596/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:06:57.282046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:57.378067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564febd-1a08-40b1-98ff-aa2f5cc05cca None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:57.412027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:57.412634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:57.412911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:57.413182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:57.413891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:57.414198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:57.416116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:57.424318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 600/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:06:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:57.430918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:57.466872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:57.532261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de56af2-a0ec-4f09-b5fa-d27684541374 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:57.567732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:57.568392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:57.568675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:57.568944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] PATH_INFO: `/v3/users/c9a0c30e88ea490cb228e9d27a6b78be` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:57.570549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a0c30e88ea490cb228e9d27a6b78be)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:57.572151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:57.573739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 597/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Mon Aug 29 21:06:57 2022] GET /identity/v3/users/c9a0c30e88ea490cb228e9d27a6b78be => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:06:57.579772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:57.615374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:57.681930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a825-e5d5-4fe6-8375-755b27d2d23f None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:57.716859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:57.717479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:57.717768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:57.718158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:57.793731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:57.795204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:57.833524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:57.902324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 601/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:06:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:06:57.909695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:58.030176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c191d3-cd08-4d0a-9429-710bdc8d4349 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:58.032756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:58.065168 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc5e14-c3d0-4a43-9c83-c340e4d75db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:58.065801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5e14-c3d0-4a43-9c83-c340e4d75db6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:58.066091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5e14-c3d0-4a43-9c83-c340e4d75db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:58.066361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5e14-c3d0-4a43-9c83-c340e4d75db6 None admin] PATH_INFO: `/v3/projects/68a6fc3b0be04272bafca71183f4f351` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:58.067470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc5e14-c3d0-4a43-9c83-c340e4d75db6 None admin] RBAC: Authorizing `identity:get_project(project_id=68a6fc3b0be04272bafca71183f4f351)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:58.069003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc5e14-c3d0-4a43-9c83-c340e4d75db6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:58.070649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 598/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Mon Aug 29 21:06:57 2022] GET /identity/v3/projects/68a6fc3b0be04272bafca71183f4f351 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:06:58.076794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:06:58.176191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95510-e3f2-4cf2-8316-30c770a5e7dd None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:06:58.209675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:06:58.210270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:06:58.211045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:06:58.211335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:06:58.212090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=68a6fc3b0be04272bafca71183f4f351, user.id=c9a0c30e88ea490cb228e9d27a6b78be) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:06:58.212402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:06:58.213831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbceec0d-958c-43ac-a8a7-4f30049b646f None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:06:58.216680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:06:58.219968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 602/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:06:58 2022] GET /identity/v3/role_assignments?user.id=c9a0c30e88ea490cb228e9d27a6b78be&scope.project.id=68a6fc3b0be04272bafca71183f4f351&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:07:08.064415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=14 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:26.234231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff314b-fff0-4fc1-bfca-ef5a0c0dcde2 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:26.235007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff314b-fff0-4fc1-bfca-ef5a0c0dcde2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:26.235214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff314b-fff0-4fc1-bfca-ef5a0c0dcde2 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:26.237840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 599/1201] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:08:26.245058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d719874-fb12-4071-b233-550da7884d40 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:26.245608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d719874-fb12-4071-b233-550da7884d40 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:26.245608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d719874-fb12-4071-b233-550da7884d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:26.265680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7d719874-fb12-4071-b233-550da7884d40 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:26.266886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7d719874-fb12-4071-b233-550da7884d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:26.367337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 603/1202] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:08:26.650993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff314b-fff0-4fc1-bfca-ef5a0c0dcde2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:26.828145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff314b-fff0-4fc1-bfca-ef5a0c0dcde2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:26.903335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:26.904375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:26.904880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:26.905381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:26.924757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:26.927746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:26.952860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:26.963922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:26.965203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:26.965203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:26.965203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:26.965203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 600/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:26.965533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d719874-fb12-4071-b233-550da7884d40 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:27.132379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d719874-fb12-4071-b233-550da7884d40 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:27.181645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:27.188856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:27.189772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:27.190116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:27.190427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:27.191156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:27.191509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:27.193128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f5bc4-ab6a-44ce-a20d-f1f43adbaa89 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:27.206808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 604/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:27.215702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:27.284924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:27.342185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec07fe-1f8c-4f65-9643-956cc418571e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:27.384811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f558f1-f9e7-4238-a5c7-f2c8fb871517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:27.385446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f558f1-f9e7-4238-a5c7-f2c8fb871517 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:27.385730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f558f1-f9e7-4238-a5c7-f2c8fb871517 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:27.386010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f558f1-f9e7-4238-a5c7-f2c8fb871517 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:27.386679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f558f1-f9e7-4238-a5c7-f2c8fb871517 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:27.388356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f558f1-f9e7-4238-a5c7-f2c8fb871517 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:27.689109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 601/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Mon Aug 29 21:08:27 2022] POST /identity/v3/users => generated 251 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:08:29.763221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58d345-e0f0-4400-b674-09795f267809 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:29.764096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58d345-e0f0-4400-b674-09795f267809 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:29.764096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58d345-e0f0-4400-b674-09795f267809 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:29.765184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 605/1206] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:08:29.771849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:29.772409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:29.772888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:29.802152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:29.804213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:29.908578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 602/1207] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:08:30.199272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58d345-e0f0-4400-b674-09795f267809 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:30.322460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58d345-e0f0-4400-b674-09795f267809 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:30.367509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:30.368359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:30.368918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:30.368918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:30.381833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:08:30.387363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:30.394049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:30.395603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 606/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:08:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:30.396996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:30.442939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:30.525011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc060-5bcf-4d5f-8f41-b2423ca9b925 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:30.571587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:30.572412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:30.573384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:30.573748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:30.574650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:30.575058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:30.576664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:30.587023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 603/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:08:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:30.596842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:30.692822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:30.765209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410b1c4-ce48-4d45-b9ee-24fca46ed06b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:30.823363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:30.824373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:30.824738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:30.825087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:30.838659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:30.840441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:30.856879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:30.859776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:30.861202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:30.861202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:30.861202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:30.861202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 607/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:30.865912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:31.003312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:31.007659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633c1b2-66d8-4e6d-b377-074791d5a3ea None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:31.050905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:31.051609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:31.051796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:31.052109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:31.052767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:31.053134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:31.056080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:31.071692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 604/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:31.078163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:31.230096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f2b10-0196-4f95-b540-c0afcfa83907 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:31.231861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:31.275576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:31.276766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:31.277082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:31.277419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:31.287094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:31.292679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:08:31.302171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:31.304156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:31.305979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 608/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:31.309781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:31.431151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7fa6b-a857-452b-924f-6e3617ff840e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:31.480372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:31.481540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:31.481540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:31.481540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:31.482263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:31.482571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:31.484218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:31.505619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 605/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Mon Aug 29 21:08:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:31.512690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:31.692681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6933e51-36dd-4821-ae43-b49f3ca4ff12 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:31.739233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:31.743922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:31.743922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:31.743922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:31.788343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:31.843414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:31.846453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:32.036392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 609/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:08:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:08:32.043684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:32.071941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:32.196088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cde0d7-84fa-4022-9d5c-c38f703db824 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:32.239812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:32.240518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:32.240802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:32.241076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:32.254455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:32.256087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:32.278968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:32.281359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:32.283456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:32.283456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:32.283456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:32.283456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 606/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:32.294622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:32.381350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:32.441325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a46eb-aa43-4b93-9e97-9f9b71f6607d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:32.496826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:32.497790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:32.498247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:32.498687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:32.499825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:32.500347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:32.502803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:32.525156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 610/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:32.533576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:32.624709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:32.661333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bbb8-8f6c-40ba-8f02-81274de1e82e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:32.702576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:32.703307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:32.703596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:32.703876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:32.712114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:32.714162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:08:32.726788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:32.728445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:32.730308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 607/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:08:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:32.734367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:32.857840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993c739-3969-43b2-9bda-0a5130bbf60e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:32.888947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:32.901865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:32.902482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:32.902762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:32.903057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:32.903756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:32.904084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:32.905832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908e40-399d-454b-b378-5dc6b7ae1a79 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:32.918769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 611/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:08:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:32.925258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:33.035555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6831627-e401-42cd-830f-07fcb1307232 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:33.072751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:33.079869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:33.080545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:33.080821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:33.081093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:33.081833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=5c39ceb5a1fb48328b9ca050172da36e) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:33.082138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:33.083611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3880f7b-1425-4308-abfe-09bb44a7fd89 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:33.090448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 608/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:08:32 2022] GET /identity/v3/role_assignments?user.id=5c39ceb5a1fb48328b9ca050172da36e&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:35.285387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60de26-3d2d-460c-a809-5c8de71656f0 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:35.285922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60de26-3d2d-460c-a809-5c8de71656f0 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:35.285922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60de26-3d2d-460c-a809-5c8de71656f0 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:35.287304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 612/1220] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:08:35.293370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:35.293907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:35.294005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:35.315868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:35.317144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:35.392310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 609/1221] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:08:35.662699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60de26-3d2d-460c-a809-5c8de71656f0 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:35.798870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60de26-3d2d-460c-a809-5c8de71656f0 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:35.845887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:35.846549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:35.846897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:35.847194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:35.861140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:08:35.866330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:35.867466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:35.874711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 613/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:08:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:35.875854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:36.004843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed2840-42a4-4611-98f2-e364eba2c1c1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:36.046858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:36.073510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:36.074231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:36.074674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:36.075188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:36.076159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] RBAC: Adding query filter params (name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:36.076473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:36.078146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:36.091486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 610/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:36.099550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:36.155729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:36.236556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9f560-0532-43c0-9b60-b5e0cf3155a5 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:36.286076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:36.287158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:36.287704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:36.288162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:36.308429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:36.311078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:36.334617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:36.338366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:36.340068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:36.340068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:36.340068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:36.340068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 614/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:36.346679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:36.511015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af957b-839a-45e4-8d88-7a461ba6269e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:36.553944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:36.554598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:36.554922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:36.555240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:36.556158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:36.556533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:36.558080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:36.573445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 611/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:36.581733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:36.726344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:36.735353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6073c9-2732-461b-a20a-a6b51267edf7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:36.784220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:36.785250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:36.785675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:36.786099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:36.796995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:36.798608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:08:36.812586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:36.813739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:36.814725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 615/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:36.824472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:36.911877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:36.979324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c4dac-9579-4d61-b215-bb7018f30563 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:37.022558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:37.023657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:37.024185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:37.029258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:37.029258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:37.029258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:37.029873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:37.070119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 612/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Mon Aug 29 21:08:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:37.078014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:37.237453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936d26-18c2-48d3-93bf-4532d1e2a268 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:37.286770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:37.287791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:37.287986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:37.288251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:37.302916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:37.305068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:37.325440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:37.327175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:37.328520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:37.328520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:37.328520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:37.328520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 616/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:37.336280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:37.462393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696bba-8e57-499f-ab1a-700532c94278 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:37.465274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:37.506499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:37.507512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:37.508038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:37.508515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:37.509712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:37.510242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:37.512741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:37.535572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 613/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:37.541860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:37.552802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:37.711819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695667d4-0617-491c-b188-9c9f6ac5dc70 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:37.765384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:37.766785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:37.767275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:37.767744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:37.781441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:37.784346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:08:37.807358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:37.809056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:37.810726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 617/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:37.816209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:37.969243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93b55-a871-4bce-996b-77c0deb9a9a6 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:38.010678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:38.012905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:38.012905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:38.012905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:38.012905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:38.013983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:38.015316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7fc99-ea46-4143-8b6c-4aedabfa5c6e None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:38.032337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 614/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:08:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:38.039224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:38.134712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:38.186694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8079bc3-8bb9-46db-82fb-98ed99646c32 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:38.226761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd72323-473a-4479-bb45-580548859735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:38.227407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72323-473a-4479-bb45-580548859735 None admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:38.227730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72323-473a-4479-bb45-580548859735 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:38.228086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72323-473a-4479-bb45-580548859735 None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/5c39ceb5a1fb48328b9ca050172da36e/roles/f20b7b217de247bd9f65721043fd0288` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:38.247033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd72323-473a-4479-bb45-580548859735 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=5c39ceb5a1fb48328b9ca050172da36e, role_id=f20b7b217de247bd9f65721043fd0288)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:38.249621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd72323-473a-4479-bb45-580548859735 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:38.396703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 618/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:08:38 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/5c39ceb5a1fb48328b9ca050172da36e/roles/f20b7b217de247bd9f65721043fd0288 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:08:40.562601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a0f9-e45c-41fc-b2d2-ef67426b9730 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:40.563536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a0f9-e45c-41fc-b2d2-ef67426b9730 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:40.563536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88a0f9-e45c-41fc-b2d2-ef67426b9730 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:40.565287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 615/1233] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:08:40.572624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:40.573235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:40.573501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:40.598815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:40.600221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:40.695844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 619/1234] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:08:40.980219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a0f9-e45c-41fc-b2d2-ef67426b9730 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:41.028834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:41.137177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88a0f9-e45c-41fc-b2d2-ef67426b9730 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:41.199633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:41.200419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:41.200844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:41.201221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:41.218398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:08:41.223691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:41.225137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:41.226304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 616/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Mon Aug 29 21:08:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:41.233206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:41.308483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:41.374487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81129889-fa85-4e63-bcf9-d256ce89345c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:41.418011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:41.418995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:41.419453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:41.419937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:41.420908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:41.421931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:41.423597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:41.434702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 620/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Mon Aug 29 21:08:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:41.443150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:41.564017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:41.581275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1279659-ccc7-4098-8590-9444474eff74 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:41.619338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:41.621839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:41.621839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:41.621839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:41.632570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:41.634182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:41.653129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:41.655249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:41.657073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:41.657073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:41.657073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:41.657073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 617/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:41.665768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:41.865613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7e71-294a-49a4-b4bc-78a8971edc1d None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:41.884837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:41.930890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:41.931952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:41.932455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:41.932948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:41.934124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:41.934678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:41.937802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:41.961494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 621/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:41.969789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:42.113270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c853e0-f71b-4a44-a91b-f07babfbc63a None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:42.165462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:42.166124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:42.166402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:42.166677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:42.170349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:42.176540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:42.178055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:08:42.191875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:42.194092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:42.196194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 618/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:42.204264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:42.391747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7203-a608-4cf3-81b5-5531b95d4c33 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:42.449631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:42.450253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:42.450538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:42.450811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:42.451562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:42.451862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:42.453950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:42.476520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 622/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Mon Aug 29 21:08:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:42.483387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:42.639479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c9759-7043-4bfe-b534-c34a7666dee2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:42.689942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:42.690520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:42.690828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:42.691318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:42.778567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:42.786595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:42.788333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:42.907230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 619/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:08:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 21:08:42.913604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:42.965772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:43.071764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a5645-8ad2-4815-8767-9da412808d93 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:43.115912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:43.117103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:43.117674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:43.118012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:43.134819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:43.136376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:43.154830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:43.157012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:43.158953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:43.158953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:43.158953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:43.158953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 623/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:43.162987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:43.295045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:43.306546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f2bd3-78d0-4fb5-87e4-2da74cf93e3d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:43.366972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:43.367666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:43.368024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:43.368349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:43.369198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:43.369588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:43.371160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:43.384991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 620/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:43.391315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:43.495110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:43.519339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df49d3-5f72-4bcf-b12e-5da3c642b2e1 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:43.577065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:43.578155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:43.578754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:43.579200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:43.592382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:43.595718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:08:43.616905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:43.618247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:43.619262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 624/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:08:43.628602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:43.764606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b19f-ba21-48bd-af5d-9cf656b32a85 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:43.807402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:43.808118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:43.808450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:43.808768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:43.809770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:43.810339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:43.813283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b67e9-4758-4c55-adb1-0c4f80d63939 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:43.830580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 621/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:08:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:43.837551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:43.976032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961453a-5c3d-46df-807a-a097ccb63176 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:44.018436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:44.019054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:44.019333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:44.019605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:44.020391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] RBAC: Adding query filter params (role.id=f20b7b217de247bd9f65721043fd0288, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=5c39ceb5a1fb48328b9ca050172da36e) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:44.021218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:44.022593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98c97b-366e-4d27-bc88-8b682c534e50 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:44.029587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 625/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:08:43 2022] GET /identity/v3/role_assignments?user.id=5c39ceb5a1fb48328b9ca050172da36e&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=f20b7b217de247bd9f65721043fd0288 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:46.152459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e724b8-0515-419b-94f8-51b739fc218d None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:46.153045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e724b8-0515-419b-94f8-51b739fc218d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:46.153045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e724b8-0515-419b-94f8-51b739fc218d None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:46.154455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 622/1247] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:08:46.163365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:46.163939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:46.169206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:46.171092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:46.185214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:46.186431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:46.274995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 626/1248] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:08:46.553680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e724b8-0515-419b-94f8-51b739fc218d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:46.575706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:46.674186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e724b8-0515-419b-94f8-51b739fc218d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:46.720100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:46.720798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:46.721090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:46.721361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:46.732811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:08:46.737811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:46.739166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:46.740856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 623/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:46.748799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:46.867374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ca23e-4487-4e0f-bffa-0c73121e4233 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:46.901410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:46.906792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:46.907420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:46.907734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:46.908084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:46.908852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:46.909158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:46.910619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:46.920358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 627/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:08:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:46.927219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:47.038808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:47.068637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4895b-62b2-426d-99ef-e0d8fe1691f7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:47.107875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:47.108549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:47.108842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:47.109172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:47.121765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:47.123394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:47.141847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:47.144877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:47.147083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:47.147083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:47.147083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:47.147083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 624/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:47.151192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:47.270795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad225b3-3d8d-49ce-9be9-355ffec165d0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:47.312521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:47.313142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:47.313485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:47.313737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:47.314492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:47.314800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:47.316246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:47.330923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 628/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:47.337976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:47.458912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34552b-62d9-4cff-95a5-528ae52aed89 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:47.490016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:47.498755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:47.499403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:47.499681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:47.499982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:47.509982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:47.511480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:08:47.524874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:47.526839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:47.528811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 625/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:47.532979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:47.595929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:47.649508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1a224-c161-4e1c-802c-ece279ac6b1b None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:47.687261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:47.688822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:47.689215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:47.690126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:47.691098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:47.691526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:47.693163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:47.718142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 629/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Mon Aug 29 21:08:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:47.724802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:47.860051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89a18e-b7e3-486f-9e48-1cadcee70c76 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:47.921955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:47.922695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:47.923023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:47.923374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:47.990185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:48.004443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:48.005989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:48.139809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 626/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:08:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:08:48.147274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:48.306017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eec70-5c31-46d8-a932-a2b3539ee1a0 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:48.320568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:48.343842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:48.344511 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:48.344875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:48.345238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:48.358588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:48.360276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:48.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:48.378555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:48.380376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:48.380376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:48.380376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:48.380376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 630/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:48.384296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:48.483019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:48.499313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416cdab-93dd-4517-aee3-b0b9b8e94eaa None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:48.541531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:48.542152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:48.542478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:48.542829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:48.543474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:48.543777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:48.545205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:48.556826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 627/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:48.562689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:48.699673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0eed2-9e65-4981-81b6-7aac769d2402 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:48.743912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:48.744708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:48.745030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:48.745344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:48.754947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:48.756679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:08:48.771303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:48.773065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:48.774569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 631/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:48.780075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:48.907553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46e531-d420-4e5a-a88a-89fc751583a1 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:48.950651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:48.951275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:48.951596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:48.951872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:48.952606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:48.952986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:48.954549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24960ebd-9c9d-485b-84b3-bdd53a33ef16 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:48.969259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 628/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:08:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:48.976171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:49.007046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:49.100402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc26bff-41a0-48af-af7e-5f106219e501 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:49.161748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:49.162306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:49.162630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:49.163024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:49.163870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=5c39ceb5a1fb48328b9ca050172da36e) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:49.164223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:49.165973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1906f5-576b-47ae-9553-fd33314f0272 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:49.172777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 632/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:08:48 2022] GET /identity/v3/role_assignments?user.id=5c39ceb5a1fb48328b9ca050172da36e&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:51.463857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8bf7-d3d8-4133-8931-d501433700fd None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:51.464699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8bf7-d3d8-4133-8931-d501433700fd None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:51.464699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a8bf7-d3d8-4133-8931-d501433700fd None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:51.465717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 629/1261] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:08:51.474209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:51.474915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:51.475112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:51.499236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:51.500762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:51.579856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 633/1262] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:08:51.858556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8bf7-d3d8-4133-8931-d501433700fd None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:51.877215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:52.038880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a8bf7-d3d8-4133-8931-d501433700fd None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:52.080471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:52.081125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:52.081408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:52.081684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:52.094202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:08:52.099823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:52.101677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:52.103439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 630/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:52.107639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:52.128647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:52.225622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add6aed-f524-40b9-b531-f271b9b8f496 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:52.266548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:52.267304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:52.267702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:52.268054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:52.268779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:52.269124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:52.271088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:52.281418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 634/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:08:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:52.288764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:52.364541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:52.408368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a4385-e205-4d52-b54a-38791938660e None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:52.444470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:52.445061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:52.445343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:52.445618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:52.458128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:52.459603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:52.475031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:52.477780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:52.479826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:52.479826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:52.479826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:52.479826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 631/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:52.484114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:52.577580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:52.599447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf651a3-fc71-4cef-b809-a942634646c7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:52.634656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:52.635327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:52.635585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:52.635856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:52.636534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:52.636840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:52.638229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:52.651767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 635/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:08:52.658035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:52.764052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c824102-24bb-45f9-baac-8d00af7a7fe2 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:52.785752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:52.804413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66574a-0019-451d-a973-779683727145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:52.805061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66574a-0019-451d-a973-779683727145 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:52.805340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66574a-0019-451d-a973-779683727145 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:52.805614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66574a-0019-451d-a973-779683727145 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:52.820073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66574a-0019-451d-a973-779683727145 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66574a-0019-451d-a973-779683727145 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-3f66574a-0019-451d-a973-779683727145 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:52.854426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:52.855433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:52.856442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 632/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:52.857185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:52.978532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ebc6-0a8a-4f83-a711-863fd78af364 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:53.020401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:53.020401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:53.020401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:53.020401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:53.020816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:53.020816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:53.020816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:53.030339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:53.040100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 636/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Mon Aug 29 21:08:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:53.045897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66574a-0019-451d-a973-779683727145 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:53.169118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66574a-0019-451d-a973-779683727145 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:53.219686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:53.220646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:53.220926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:53.221289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:53.234373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:53.235851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:53.251717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:53.256506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:53.257429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:53.257429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:53.257429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:53.257429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 633/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:53.260284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:53.365849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0bcc9-f2dd-4b9a-92d9-088497c36327 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:53.401779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:53.402406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:53.402683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:53.402954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:53.403606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:53.403936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:53.405327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:53.417849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 637/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:53.424843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:53.461503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:53.547650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4b796-e30b-4cc5-8393-047577c149b7 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:53.586211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:53.586994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:53.587326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:53.587696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:53.595071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:53.596569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:08:53.608903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:53.610052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:53.611006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 634/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:08:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:53.616340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:53.688788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:53.733623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bb967-7d3d-4bba-af2a-fbfa43366628 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:53.769064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:53.769882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:53.770695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:53.770971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:53.771688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:53.772176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:53.773932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93f139-d2ae-47f8-82f9-9fd6f45a1842 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:53.789610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 638/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:53.798296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:53.927382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:53.939794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c49972-42c8-4d06-a315-da87fba82dd8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:53.996212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dbb95-cc99-4b62-bd82-fa050c70c8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:53.996995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dbb95-cc99-4b62-bd82-fa050c70c8ca None admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:53.997550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dbb95-cc99-4b62-bd82-fa050c70c8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:53.998050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dbb95-cc99-4b62-bd82-fa050c70c8ca None admin] PATH_INFO: `/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/5c39ceb5a1fb48328b9ca050172da36e/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:54.000740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dbb95-cc99-4b62-bd82-fa050c70c8ca None admin] RBAC: Authorizing `identity:create_grant(project_id=0ffba55e45324a748e9fcb000ff276ec, user_id=5c39ceb5a1fb48328b9ca050172da36e, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:54.002363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dbb95-cc99-4b62-bd82-fa050c70c8ca None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:54.113693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 635/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Mon Aug 29 21:08:53 2022] PUT /identity/v3/projects/0ffba55e45324a748e9fcb000ff276ec/users/5c39ceb5a1fb48328b9ca050172da36e/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:08:56.161695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c2a7-3d40-474d-a8f6-1df8c32258ca None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:56.162241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c2a7-3d40-474d-a8f6-1df8c32258ca None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:56.162369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5c2a7-3d40-474d-a8f6-1df8c32258ca None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:56.164410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 639/1274] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:08:56.173034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:56.175745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:56.175745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:56.194562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:08:56.195796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:08:56.272489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 636/1275] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:08:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:08:56.550945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c2a7-3d40-474d-a8f6-1df8c32258ca None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:56.671040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5c2a7-3d40-474d-a8f6-1df8c32258ca None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:56.702378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:56.730353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:56.731245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:56.731640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:56.732079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:56.748881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 21:08:56.759266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:56.760591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:56.761767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 640/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Mon Aug 29 21:08:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:56.768620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:56.861402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:56.881070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b2211-272e-4990-9f6a-0e8739188227 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:56.920388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:56.921004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:56.921281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:56.921548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:56.922210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:56.922511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:56.924042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:56.933507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 637/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Mon Aug 29 21:08:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:56.939665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:57.056473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35820056-7725-4faa-8684-2d9efca4ee05 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:57.096179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:57.097257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:57.097768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:57.098276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:57.118334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:57.121035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:57.150041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:57.154976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:57.156699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:57.156699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:57.156699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:57.156699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 641/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:57.162563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:57.289185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801a553-7426-45ec-8d43-c88db2fcd70c None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:57.327120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:57.327918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:57.328475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:57.328832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:57.329826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:57.331256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:57.332767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:57.346227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 638/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:57.352681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:57.408379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:57.494883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78289020-bb77-41ba-a6de-a11687066e76 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:57.532710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:57.533359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:57.533637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:57.533910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:57.543510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:57.545056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 21:08:57.557432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:57.559219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:08:57.561122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 642/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:57.564956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:57.570739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:57.683864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62215c82-6fe0-41c4-be19-cdb2328d4652 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:57.747280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:57.748362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:57.748854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:57.749333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:57.750713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:57.751453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:57.754214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:57.789497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 639/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Mon Aug 29 21:08:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:57.795719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:57.900649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:57.910655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4f71c-cb3e-42bf-bfd0-7fe3618e9333 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:57.947643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:57.948216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:57.948497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:57.948768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:58.025468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:58.026945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:58.138125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:58.151319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 643/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Mon Aug 29 21:08:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 21:08:58.157605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:58.239844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:58.309191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac78d-f8f3-4794-bf67-3abbcb6faeb8 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:58.370543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:58.371630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:58.372144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:58.372652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:58.391364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:58.394002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 21:08:58.423006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:58.424419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 21:08:58.425409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 21:08:58.425409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 21:08:58.425409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:58.425409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 640/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:58.433801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:58.590522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8a097-b3bb-4433-8c6a-1997ac40481a None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:58.629404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:58.630023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:58.630302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:58.630576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:58.631837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:58.632233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:58.634041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:58.646156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 644/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Mon Aug 29 21:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:58.653022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:58.776459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfcee8-6f94-43e5-a359-7b690dced471 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:58.795685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:58.821452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:58.822271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:58.822665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:58.823060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:58.831690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:58.833394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 21:08:58.848094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application with Lock( Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:08:58.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: ERROR keystone.server.flask.application Aug 29 21:08:58.851570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 641/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Mon Aug 29 21:08:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:08:58.856073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:58.918645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:08:58.973203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885aa2f-d5bc-447d-8eb9-5a1a4dac980e None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:59.010937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:59.011553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:59.011833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:59.012255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:59.012825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:59.013131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:59.014569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3538b2-8160-43b4-9d8b-887b9d4308cf None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:59.027718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 645/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Mon Aug 29 21:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:08:59.038051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:08:59.155347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c123da-368c-4e4c-8e62-c3cabb6fd50d None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:08:59.197943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:08:59.198580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:08:59.198863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:08:59.202647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:08:59.202647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] RBAC: Adding query filter params (role.id=d8cbc03a891c4dbfb863c52fbeaddbc3, scope.project.id=0ffba55e45324a748e9fcb000ff276ec, user.id=5c39ceb5a1fb48328b9ca050172da36e) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:08:59.202647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:08:59.202647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a956b3a-9e97-4514-98dd-31c97e2eb7a3 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:08:59.209041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 642/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Mon Aug 29 21:08:59 2022] GET /identity/v3/role_assignments?user.id=5c39ceb5a1fb48328b9ca050172da36e&scope.project.id=0ffba55e45324a748e9fcb000ff276ec&role.id=d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:09:01.332807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674987ad-9410-475e-be6b-939d37299dd5 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:01.333340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674987ad-9410-475e-be6b-939d37299dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:01.333340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674987ad-9410-475e-be6b-939d37299dd5 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:01.334713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 646/1288] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:09:01.341843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:01.342564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:01.342564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:01.366178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:09:01.367454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:09:01.404773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:01.444662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 643/1289] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:09:01.716280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674987ad-9410-475e-be6b-939d37299dd5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:01.856880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674987ad-9410-475e-be6b-939d37299dd5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:01.878133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:01.913721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:01.915033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:01.915361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:01.915681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:01.916416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:01.919421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:09:01.928159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:09:01.932760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:01.934940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 647/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:09:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:09:01.936682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:02.050094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c03fe1-9e3f-4cb8-b7b3-627830545b1f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:02.088101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:02.088803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:02.089123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:02.089439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:02.090130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:09:02.090536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:02.091980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4e476-2dc8-44d7-a964-6fa7c40d83bd None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:02.097536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 644/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Mon Aug 29 21:09:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:09:02.108524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:02.234263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28473696-4f63-4cfa-add2-412cd9b8e275 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:02.275141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:02.275762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:02.276065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:02.276339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] PATH_INFO: `/v3/services` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:02.276978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:09:02.277276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:02.278712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311f2db-aa39-4ea6-8af6-b638c132793c None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:02.283998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 648/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Mon Aug 29 21:09:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:09:04.244378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1501-35db-48c6-9539-a242696d9ae7 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:04.244967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1501-35db-48c6-9539-a242696d9ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:04.244967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1501-35db-48c6-9539-a242696d9ae7 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:04.247943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 645/1293] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:09:04.256035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbff13f-7e45-4c72-82e5-1871adba64cf None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:04.256590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbff13f-7e45-4c72-82e5-1871adba64cf None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:04.256929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbff13f-7e45-4c72-82e5-1871adba64cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:04.287480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-bbbff13f-7e45-4c72-82e5-1871adba64cf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:09:04.289388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bbbff13f-7e45-4c72-82e5-1871adba64cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:09:04.341916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:04.384795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 649/1294] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:09:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:09:04.676303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e1501-35db-48c6-9539-a242696d9ae7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:04.711038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:04.788376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e1501-35db-48c6-9539-a242696d9ae7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:04.827118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151494-1c76-40b9-ab73-92a4c23d8047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:04.827783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151494-1c76-40b9-ab73-92a4c23d8047 None admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:04.828092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151494-1c76-40b9-ab73-92a4c23d8047 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:04.828370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151494-1c76-40b9-ab73-92a4c23d8047 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:04.829054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151494-1c76-40b9-ab73-92a4c23d8047 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:04.830505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151494-1c76-40b9-ab73-92a4c23d8047 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:04.993593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 646/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Mon Aug 29 21:09:04 2022] POST /identity/v3/services => generated 246 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:09:06.996120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323ac17-f6b6-4178-a693-a862da373a1c None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:06.997013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323ac17-f6b6-4178-a693-a862da373a1c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:06.997013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323ac17-f6b6-4178-a693-a862da373a1c None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:06.998481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 650/1296] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:09:07.008430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:07.009224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:07.009224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:07.029295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:09:07.030907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:09:07.119206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 647/1297] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:09:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 21:09:07.606040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323ac17-f6b6-4178-a693-a862da373a1c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:07.731035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323ac17-f6b6-4178-a693-a862da373a1c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:07.776390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:07.777113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:07.777461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:07.777769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:07.778509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:07.779807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:09:07.789361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:09:07.791924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:07.793625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 651/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:09:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 21:09:07.799508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:07.822564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:07.912235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44ac7-33ca-4a68-94cd-2834b3c2d316 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:07.949667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:07.950341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:07.950623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:07.950989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:07.951651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:09:07.951981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:07.953410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:07.959207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 648/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Mon Aug 29 21:09:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:09:07.966454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:08.002290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:08.078407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b489c5-f90e-442b-b1bb-a4851cbe8cf8 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:08.123211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:08.124033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:08.124425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:08.124800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:08.125672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] RBAC: Adding query filter params (interface=public, service_id=5ea338403e6b45909d43a118b389552b, region_id=RegionOne) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:09:08.126083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:08.127800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa93e0-4ea6-46f5-87dc-2866b3d57151 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:08.134098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 652/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Mon Aug 29 21:09:07 2022] GET /identity/v3/endpoints?service_id=5ea338403e6b45909d43a118b389552b&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:09:08.140763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:08.244760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b6df-30f2-4076-b159-1ae4a65b00e9 None admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:08.285755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4daa1b2-4f62-45e2-bf2d-4de77734d912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:08.286372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daa1b2-4f62-45e2-bf2d-4de77734d912 None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:08.286651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daa1b2-4f62-45e2-bf2d-4de77734d912 None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:08.286922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4daa1b2-4f62-45e2-bf2d-4de77734d912 None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:08.287560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4daa1b2-4f62-45e2-bf2d-4de77734d912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:08.288970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4daa1b2-4f62-45e2-bf2d-4de77734d912 None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:08.296496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 649/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Mon Aug 29 21:09:08 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:09:08.461105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:10.317412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f1281-e74b-4b84-ad10-cfd22dc5ee0a None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:10.317956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f1281-e74b-4b84-ad10-cfd22dc5ee0a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:10.317956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f1281-e74b-4b84-ad10-cfd22dc5ee0a None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:10.319448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 653/1302] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:09:10.325433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:10.326209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:10.326366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:10.346849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:09:10.348118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:09:10.422251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 650/1303] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:09:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 21:09:10.693254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f1281-e74b-4b84-ad10-cfd22dc5ee0a None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:10.806877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:10.814884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f1281-e74b-4b84-ad10-cfd22dc5ee0a None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:10.851352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:10.852063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:10.852325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:10.852603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:10.853199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:10.855197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 21:09:10.864820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 21:09:10.866738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application with Lock( Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self._enter() Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application return self.creator() Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application created_value = creator( Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: ERROR keystone.server.flask.application Aug 29 21:09:10.868637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 654/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Mon Aug 29 21:09:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 21:09:10.875408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:10.957628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:10.982676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4759cb4-36b4-4145-8a9e-6ff13dbfcf96 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:11.018232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:11.018860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:11.019148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:11.019423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] PATH_INFO: `/v3/services` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:11.020099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:09:11.020406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:11.021795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d20a3-fb0b-4e8e-a27b-7c069d74fccb None admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:11.028387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 651/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Mon Aug 29 21:09:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:09:11.037881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:09:11.164527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3de329-0817-4737-8453-2459269dd7a7 None admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:09:11.200967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eb8b9-b014-455e-8f7b-c1247eaccbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:09:11.201585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb8b9-b014-455e-8f7b-c1247eaccbf0 None admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:09:11.201863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb8b9-b014-455e-8f7b-c1247eaccbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:09:11.202132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb8b9-b014-455e-8f7b-c1247eaccbf0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:09:11.202833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771eb8b9-b014-455e-8f7b-c1247eaccbf0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:09:11.204341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771eb8b9-b014-455e-8f7b-c1247eaccbf0 None admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:09:11.343272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 655/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:09:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:09:21.046037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=30 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:09:21.337836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:06.249423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cbc7-0d18-4c9e-a508-367c1252a350 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:06.250335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cbc7-0d18-4c9e-a508-367c1252a350 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:06.250779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cbc7-0d18-4c9e-a508-367c1252a350 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:06.252608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 652/1307] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:06.257883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120e9f0-60b8-436c-a23d-dd138fff3cef None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:06.258864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120e9f0-60b8-436c-a23d-dd138fff3cef None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:06.259401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120e9f0-60b8-436c-a23d-dd138fff3cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:06.413662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c120e9f0-60b8-436c-a23d-dd138fff3cef None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:06.414889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c120e9f0-60b8-436c-a23d-dd138fff3cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:06.602155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 656/1308] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:06.940260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc68136-2551-4550-9568-1f3132ec47bf None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:06.941111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc68136-2551-4550-9568-1f3132ec47bf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:06.941740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc68136-2551-4550-9568-1f3132ec47bf None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:06.944001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 653/1309] 10.222.0.65 () {54 vars in 965 bytes} [Mon Aug 29 21:18:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:06.952013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345eaca-5a53-4a34-83e2-0f33095d3153 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:06.952659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345eaca-5a53-4a34-83e2-0f33095d3153 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:06.953177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345eaca-5a53-4a34-83e2-0f33095d3153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:07.056426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5345eaca-5a53-4a34-83e2-0f33095d3153 None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:07.058723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5345eaca-5a53-4a34-83e2-0f33095d3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:07.084279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:07.160072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 657/1310] 10.222.0.65 () {58 vars in 1062 bytes} [Mon Aug 29 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:07.169449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc68136-2551-4550-9568-1f3132ec47bf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:07.171314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0cc68136-2551-4550-9568-1f3132ec47bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:07.305396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc68136-2551-4550-9568-1f3132ec47bf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:07.390976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:07.391739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:07.392107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:07.392460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:07.394172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:07.457862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:07.520730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:07.522395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7190559b-4a66-44b0-8c7b-afda56670686 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:07.546293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 654/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:09.860092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3190-63b7-42c6-9355-a33e732f8a24 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:09.860657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3190-63b7-42c6-9355-a33e732f8a24 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:09.860657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3190-63b7-42c6-9355-a33e732f8a24 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:09.862279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 658/1312] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:09.867698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56e8fe-85f2-4cc8-a67b-9f9099291ce8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:09.868379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56e8fe-85f2-4cc8-a67b-9f9099291ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:09.868379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56e8fe-85f2-4cc8-a67b-9f9099291ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:09.961000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6b56e8fe-85f2-4cc8-a67b-9f9099291ce8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:09.962256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6b56e8fe-85f2-4cc8-a67b-9f9099291ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:10.085726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 655/1313] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:10.371180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c3190-63b7-42c6-9355-a33e732f8a24 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:10.372876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bc1c3190-63b7-42c6-9355-a33e732f8a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:10.491466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c3190-63b7-42c6-9355-a33e732f8a24 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:10.573776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:10.577778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:10.578486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:10.578826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:10.579135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:10.581015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:10.711358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:10.712919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44c196-1bc1-4dfc-a4f5-ed0f580af989 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:10.732949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 659/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:13.402086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3cae-51cc-4d4f-9a32-058b0cb26dfb None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:13.402689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3cae-51cc-4d4f-9a32-058b0cb26dfb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:13.402689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3cae-51cc-4d4f-9a32-058b0cb26dfb None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:13.404128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 656/1315] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:13.410959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c1df0-9ec8-471f-9407-dd8881ce8847 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:13.411461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c1df0-9ec8-471f-9407-dd8881ce8847 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:13.411585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c1df0-9ec8-471f-9407-dd8881ce8847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:13.489458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9f8c1df0-9ec8-471f-9407-dd8881ce8847 None None] MFA Rules not processed for user `7e902eff1ce140c9a81aac150e114ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:13.491063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9f8c1df0-9ec8-471f-9407-dd8881ce8847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:13.557950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:13.609149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 660/1316] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:13.906019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a3cae-51cc-4d4f-9a32-058b0cb26dfb None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:13.907623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-aa2a3cae-51cc-4d4f-9a32-058b0cb26dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:13.918666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:14.032799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a3cae-51cc-4d4f-9a32-058b0cb26dfb None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:14.128374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:14.129002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:14.129280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:14.129553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:14.131239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:14.219768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:14.273653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:14.275133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db6ee5-c057-42d9-ab37-f8976aaa1857 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:14.293326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 657/1317] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:17.277950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab560fe-7eee-4749-b8f7-02129ffece0f None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:17.278491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab560fe-7eee-4749-b8f7-02129ffece0f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:17.278491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab560fe-7eee-4749-b8f7-02129ffece0f None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:17.279871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 661/1318] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:17.285184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6416e6-1b74-4ac7-89f8-8311de2ded50 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:17.285606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6416e6-1b74-4ac7-89f8-8311de2ded50 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:17.285774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6416e6-1b74-4ac7-89f8-8311de2ded50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:17.352965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ec6416e6-1b74-4ac7-89f8-8311de2ded50 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:17.354761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ec6416e6-1b74-4ac7-89f8-8311de2ded50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:17.451826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 658/1319] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:17.733284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab560fe-7eee-4749-b8f7-02129ffece0f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:17.734984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8ab560fe-7eee-4749-b8f7-02129ffece0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:17.846030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab560fe-7eee-4749-b8f7-02129ffece0f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:17.920636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:17.921243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:17.921521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:17.921788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:17.923373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:17.930803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:18.035787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:18.037178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bba9e-d852-444f-9a4e-84734fcb9496 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:18.059955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 662/1320] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:21.170024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cd01-a4c0-433a-b0d3-e727df8da65c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:21.170585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cd01-a4c0-433a-b0d3-e727df8da65c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:21.170585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59cd01-a4c0-433a-b0d3-e727df8da65c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:21.171963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 659/1321] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:21.177156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1dacb-bcee-4dc2-be23-9559c73e0ee7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:21.177581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1dacb-bcee-4dc2-be23-9559c73e0ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:21.177962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1dacb-bcee-4dc2-be23-9559c73e0ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:21.248046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-75e1dacb-bcee-4dc2-be23-9559c73e0ee7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:21.249250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-75e1dacb-bcee-4dc2-be23-9559c73e0ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:21.308502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:21.348249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 663/1322] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:21.634638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cd01-a4c0-433a-b0d3-e727df8da65c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:21.636293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4c59cd01-a4c0-433a-b0d3-e727df8da65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:21.658734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:21.756229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59cd01-a4c0-433a-b0d3-e727df8da65c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:21.834499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:21.835119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:21.835398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:21.835673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:21.837306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:21.913244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:21.958674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:21.960185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b77a08-d749-4c6b-96a5-5e68694fba19 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:21.980142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 660/1323] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:24.010579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a425c1-93d2-4e24-916f-4ae6c594edfd None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:24.011310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a425c1-93d2-4e24-916f-4ae6c594edfd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:24.011310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a425c1-93d2-4e24-916f-4ae6c594edfd None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:24.012537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 664/1324] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:24.018155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e89be7-5d53-44ee-b4aa-a9a4727d4e2d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:24.018618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e89be7-5d53-44ee-b4aa-a9a4727d4e2d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:24.018723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e89be7-5d53-44ee-b4aa-a9a4727d4e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:24.093406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-67e89be7-5d53-44ee-b4aa-a9a4727d4e2d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:24.094561 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-67e89be7-5d53-44ee-b4aa-a9a4727d4e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:24.194240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 661/1325] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:24.497310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa61385-0f5c-407c-98e1-91abb780393d None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:24.497592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa61385-0f5c-407c-98e1-91abb780393d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:24.497868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa61385-0f5c-407c-98e1-91abb780393d None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:24.499276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 665/1326] 10.222.0.65 () {54 vars in 965 bytes} [Mon Aug 29 21:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:24.506388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356098a-92e9-4fea-9b5a-3517027f6907 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:24.506848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356098a-92e9-4fea-9b5a-3517027f6907 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:24.507219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356098a-92e9-4fea-9b5a-3517027f6907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:24.541257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:24.588915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9356098a-92e9-4fea-9b5a-3517027f6907 None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:24.590131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9356098a-92e9-4fea-9b5a-3517027f6907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:24.668817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 662/1327] 10.222.0.65 () {58 vars in 1062 bytes} [Mon Aug 29 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:24.678018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa61385-0f5c-407c-98e1-91abb780393d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:24.682464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-efa61385-0f5c-407c-98e1-91abb780393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:24.792952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa61385-0f5c-407c-98e1-91abb780393d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:24.875531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:24.876178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:24.876460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:24.876760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:24.878385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:24.895670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:24.993490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:24.995391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd8dc0-f4f1-42bf-9eb0-7e8dc41e3b60 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:25.013457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 666/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:27.235076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad351d1-2e55-4198-bd1b-bbf615e579c4 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:27.235624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad351d1-2e55-4198-bd1b-bbf615e579c4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:27.235624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad351d1-2e55-4198-bd1b-bbf615e579c4 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:27.237057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 663/1329] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:27.242180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053943-581a-4ec1-9d6f-2a2bcb003094 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:27.242600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053943-581a-4ec1-9d6f-2a2bcb003094 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:27.242805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1053943-581a-4ec1-9d6f-2a2bcb003094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:27.327305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e1053943-581a-4ec1-9d6f-2a2bcb003094 None None] MFA Rules not processed for user `7e902eff1ce140c9a81aac150e114ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:27.328570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e1053943-581a-4ec1-9d6f-2a2bcb003094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:27.398821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:27.434184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 667/1330] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:27.725286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad351d1-2e55-4198-bd1b-bbf615e579c4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:27.727006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4ad351d1-2e55-4198-bd1b-bbf615e579c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:27.832262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad351d1-2e55-4198-bd1b-bbf615e579c4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:27.857048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:27.916143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:27.916677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:27.916955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:27.917227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:27.918894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:28.040761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:28.042267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbf20c-2c28-4324-8cb2-cace9bb6a228 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:28.062529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 664/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:30.624450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf5228-6e36-4cfb-ad0c-c69c194c2bd2 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:30.624999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf5228-6e36-4cfb-ad0c-c69c194c2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:30.624999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf5228-6e36-4cfb-ad0c-c69c194c2bd2 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:30.626402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 668/1332] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:30.631420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9585c-f2f4-4b9a-9b5c-69d4bbeb2969 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:30.631828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9585c-f2f4-4b9a-9b5c-69d4bbeb2969 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:30.632063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9585c-f2f4-4b9a-9b5c-69d4bbeb2969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:30.696790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:30.699350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-96e9585c-f2f4-4b9a-9b5c-69d4bbeb2969 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:30.700601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-96e9585c-f2f4-4b9a-9b5c-69d4bbeb2969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:30.796562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 665/1333] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:31.078668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf5228-6e36-4cfb-ad0c-c69c194c2bd2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:31.080275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-83bf5228-6e36-4cfb-ad0c-c69c194c2bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:31.185396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf5228-6e36-4cfb-ad0c-c69c194c2bd2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:31.265188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:31.265799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:31.266078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:31.266351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:31.267965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:31.295388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:31.376993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:31.378336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b71180-8634-49ec-a5e8-9ef09228b792 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:31.397658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 669/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:33.355185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4054-3495-4bc2-ae20-d9a3b50296d3 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:33.355711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4054-3495-4bc2-ae20-d9a3b50296d3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:33.355711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4054-3495-4bc2-ae20-d9a3b50296d3 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:33.357079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 666/1335] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:33.362314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f92f-60e6-4cef-942f-3b124b519f5b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:33.362757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f92f-60e6-4cef-942f-3b124b519f5b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:33.362850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f92f-60e6-4cef-942f-3b124b519f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:33.430164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f433f92f-60e6-4cef-942f-3b124b519f5b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:33.433085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f433f92f-60e6-4cef-942f-3b124b519f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:33.508953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:33.531829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 670/1336] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:33.816053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4054-3495-4bc2-ae20-d9a3b50296d3 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:33.817611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-288d4054-3495-4bc2-ae20-d9a3b50296d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:33.921750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4054-3495-4bc2-ae20-d9a3b50296d3 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:33.952950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:34.000195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:34.000805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:34.001081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:34.001351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:34.002935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:34.113727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:34.115168 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c73db-faf4-4f24-9276-9307197a0313 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:34.133818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 667/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:36.906276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d33f9-8b87-4c1f-82d1-445ae1b00d39 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:36.906863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d33f9-8b87-4c1f-82d1-445ae1b00d39 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:36.906970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d33f9-8b87-4c1f-82d1-445ae1b00d39 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:36.908320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 671/1338] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:36.913601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1d47f-4f0f-4ec4-9ef6-5b9d5b456776 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:36.914018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1d47f-4f0f-4ec4-9ef6-5b9d5b456776 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:36.914161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1d47f-4f0f-4ec4-9ef6-5b9d5b456776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:36.981880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-38b1d47f-4f0f-4ec4-9ef6-5b9d5b456776 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:36.983021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-38b1d47f-4f0f-4ec4-9ef6-5b9d5b456776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:36.991244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:37.082610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 668/1339] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:37.371846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d33f9-8b87-4c1f-82d1-445ae1b00d39 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:37.373496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6c1d33f9-8b87-4c1f-82d1-445ae1b00d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:37.484037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d33f9-8b87-4c1f-82d1-445ae1b00d39 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:37.559518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:37.560150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:37.560435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:37.563751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:37.563751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:37.598958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:37.674212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:37.675591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a891-4050-423a-8f11-2474a8c9de3a service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:37.694723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 672/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:42.031937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c3948-2bd3-4562-8675-8f234bcacb17 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:42.032471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c3948-2bd3-4562-8675-8f234bcacb17 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:42.032471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c3948-2bd3-4562-8675-8f234bcacb17 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:42.033754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 669/1341] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:42.038877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c0914-38bf-4410-9d8b-fe9dc66a9400 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:42.039322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c0914-38bf-4410-9d8b-fe9dc66a9400 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:42.039418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c0914-38bf-4410-9d8b-fe9dc66a9400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:42.112920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-bd7c0914-38bf-4410-9d8b-fe9dc66a9400 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:42.114132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bd7c0914-38bf-4410-9d8b-fe9dc66a9400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:42.209861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:42.221913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 673/1342] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:18:42.512517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c3948-2bd3-4562-8675-8f234bcacb17 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:42.514046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-783c3948-2bd3-4562-8675-8f234bcacb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:42.622734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c3948-2bd3-4562-8675-8f234bcacb17 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:42.662632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:42.699860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:42.700502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:42.700789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:42.701069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:42.702806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:42.820071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:42.821790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fdc4f-235b-4799-8a6a-484180b3de22 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:42.845018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 670/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:45.893362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3b1e2-a8ae-47b9-b565-c981a7a0a23a None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:45.893905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3b1e2-a8ae-47b9-b565-c981a7a0a23a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:45.893905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3b1e2-a8ae-47b9-b565-c981a7a0a23a None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:45.895297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 674/1344] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:45.901109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f4206-5958-4483-b12e-d5ab4aff2dec None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:45.901592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f4206-5958-4483-b12e-d5ab4aff2dec None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:45.901759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f4206-5958-4483-b12e-d5ab4aff2dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:45.972318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-924f4206-5958-4483-b12e-d5ab4aff2dec None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:45.973538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-924f4206-5958-4483-b12e-d5ab4aff2dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:45.993648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:46.079913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 671/1345] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:46.364894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3b1e2-a8ae-47b9-b565-c981a7a0a23a None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:46.366474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7be3b1e2-a8ae-47b9-b565-c981a7a0a23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:46.475202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3b1e2-a8ae-47b9-b565-c981a7a0a23a None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:46.555949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:46.556573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:46.556914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:46.562205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:46.562205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:46.604675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:46.673428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:46.674964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde3779-3a44-42ee-b4c7-941f40cf9fde service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:46.694126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 675/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:51.101266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90d886-12f3-466d-b1c6-6eea91bab8e3 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:51.101778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90d886-12f3-466d-b1c6-6eea91bab8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:51.101778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90d886-12f3-466d-b1c6-6eea91bab8e3 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:51.103442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 672/1347] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:51.109237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185faf0-f795-4901-843b-83593d773be7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:51.109754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185faf0-f795-4901-843b-83593d773be7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:51.109886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185faf0-f795-4901-843b-83593d773be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:51.181090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2185faf0-f795-4901-843b-83593d773be7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:51.182303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2185faf0-f795-4901-843b-83593d773be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:51.285615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:51.288324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 676/1348] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:18:51.571228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90d886-12f3-466d-b1c6-6eea91bab8e3 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:18:51.573094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bd90d886-12f3-466d-b1c6-6eea91bab8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:51.682982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90d886-12f3-466d-b1c6-6eea91bab8e3 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:18:51.733121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:51.759459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:18:51.760108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:51.760464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:51.760714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:51.762308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:51.872416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:18:51.873779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290e800-68bd-4e9e-a084-e1f134475b17 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:18:51.892474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 673/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:18:56.982748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd02f87-df0b-4205-9db1-4eecad8b6b78 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:56.984037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd02f87-df0b-4205-9db1-4eecad8b6b78 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:56.984500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd02f87-df0b-4205-9db1-4eecad8b6b78 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:56.986463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 677/1350] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:18:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:18:56.993423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687722-095b-4cf8-84c7-f61753e2183b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:18:56.993940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687722-095b-4cf8-84c7-f61753e2183b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:18:56.994039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687722-095b-4cf8-84c7-f61753e2183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:18:57.119429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-35687722-095b-4cf8-84c7-f61753e2183b None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:18:57.121657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-35687722-095b-4cf8-84c7-f61753e2183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:18:57.152894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:18:57.257188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 674/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:01.718653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d940bc5-a7a3-43ad-a434-4ca1499044a1 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:01.719215 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d940bc5-a7a3-43ad-a434-4ca1499044a1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:01.719461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d940bc5-a7a3-43ad-a434-4ca1499044a1 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:01.721243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 678/1352] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:01.727056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43141299-13e9-46f9-abf9-2475cc5e2599 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:01.727507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43141299-13e9-46f9-abf9-2475cc5e2599 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:01.727819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43141299-13e9-46f9-abf9-2475cc5e2599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:01.818569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-43141299-13e9-46f9-abf9-2475cc5e2599 None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:01.819805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-43141299-13e9-46f9-abf9-2475cc5e2599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:01.913710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:01.969738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 675/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:03.209785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fa408-a0e3-4434-b441-bd2ded458dd7 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:03.210324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fa408-a0e3-4434-b441-bd2ded458dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:03.210324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fa408-a0e3-4434-b441-bd2ded458dd7 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:03.211664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 679/1354] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:19:03.217812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266680c2-29e1-4ecf-9e41-36d48a564c5d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:03.218299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266680c2-29e1-4ecf-9e41-36d48a564c5d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:03.218398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266680c2-29e1-4ecf-9e41-36d48a564c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:03.309282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-266680c2-29e1-4ecf-9e41-36d48a564c5d None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:03.310420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-266680c2-29e1-4ecf-9e41-36d48a564c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:03.425205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 676/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:19:06.769307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def2a38-06cb-4e95-9b70-ff3e4a0f730e None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:06.769853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def2a38-06cb-4e95-9b70-ff3e4a0f730e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:06.769853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def2a38-06cb-4e95-9b70-ff3e4a0f730e None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:06.771195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 680/1356] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:06.776400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a442987-b62b-4fa6-abf3-aa0aececc120 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:06.776827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a442987-b62b-4fa6-abf3-aa0aececc120 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:06.776982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a442987-b62b-4fa6-abf3-aa0aececc120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:06.816627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:06.858745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-3a442987-b62b-4fa6-abf3-aa0aececc120 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:06.860109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3a442987-b62b-4fa6-abf3-aa0aececc120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:06.998041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 677/1357] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:07.313931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3cfb5-9b09-492e-91c5-70a4ee5248c2 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:07.314458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3cfb5-9b09-492e-91c5-70a4ee5248c2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:07.314889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3cfb5-9b09-492e-91c5-70a4ee5248c2 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:07.316628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 681/1358] 10.222.0.65 () {54 vars in 962 bytes} [Mon Aug 29 21:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:07.326374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb81c-d1be-4e86-9bed-3a35b5087c81 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:07.326745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb81c-d1be-4e86-9bed-3a35b5087c81 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:07.327095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb81c-d1be-4e86-9bed-3a35b5087c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:07.417682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c73cb81c-d1be-4e86-9bed-3a35b5087c81 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:07.418899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c73cb81c-d1be-4e86-9bed-3a35b5087c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:07.473469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:07.552900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 678/1359] 10.222.0.65 () {58 vars in 1059 bytes} [Mon Aug 29 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:19:07.557069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3cfb5-9b09-492e-91c5-70a4ee5248c2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:07.558569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-acc3cfb5-9b09-492e-91c5-70a4ee5248c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:07.720149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3cfb5-9b09-492e-91c5-70a4ee5248c2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:07.839921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:07.882137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:07.882942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:07.883294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:07.883720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:07.885333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:07.981531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:07.983686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a15c89-2b13-44df-838e-e06fe6b6cb77 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:08.003063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 682/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:10.171681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35504b72-d80f-46f2-85b6-5a5b19ec44f8 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:10.172338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35504b72-d80f-46f2-85b6-5a5b19ec44f8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:10.172338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35504b72-d80f-46f2-85b6-5a5b19ec44f8 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:10.173722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 679/1361] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:10.179331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bad0-e6b6-40e0-9a44-f4da8b8c8921 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:10.179772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bad0-e6b6-40e0-9a44-f4da8b8c8921 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:10.179864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06bad0-e6b6-40e0-9a44-f4da8b8c8921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:10.252657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ee06bad0-e6b6-40e0-9a44-f4da8b8c8921 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:10.253868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ee06bad0-e6b6-40e0-9a44-f4da8b8c8921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:10.296041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:10.364099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 683/1362] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:10.657302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35504b72-d80f-46f2-85b6-5a5b19ec44f8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:10.658669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-35504b72-d80f-46f2-85b6-5a5b19ec44f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:10.824129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35504b72-d80f-46f2-85b6-5a5b19ec44f8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:10.834968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:10.948500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:10.949057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:10.949331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:10.949606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:10.951260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:11.058915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:11.061089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85af32-34ae-4660-be95-abbcbcb319ad service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:11.086584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 680/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:13.574147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8db8c-70ff-433b-b138-929f55665f4b None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:13.574708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8db8c-70ff-433b-b138-929f55665f4b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:13.574708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8db8c-70ff-433b-b138-929f55665f4b None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:13.576375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 684/1364] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:19:13.581752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205e22c-dc49-437b-b068-eef8ba2fec4e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:13.582234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205e22c-dc49-437b-b068-eef8ba2fec4e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:13.582314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205e22c-dc49-437b-b068-eef8ba2fec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:13.599374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:13.679306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-5205e22c-dc49-437b-b068-eef8ba2fec4e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:13.681610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5205e22c-dc49-437b-b068-eef8ba2fec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:13.836756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 681/1365] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:14.144126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e72af-6a7f-4ef3-9ca1-101548c683ab None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:14.145105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e72af-6a7f-4ef3-9ca1-101548c683ab None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:14.145105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e72af-6a7f-4ef3-9ca1-101548c683ab None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:14.146570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 685/1366] 10.222.0.65 () {54 vars in 962 bytes} [Mon Aug 29 21:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:14.153758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2a054-0fb4-4426-98af-552f9af0137d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:14.154117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2a054-0fb4-4426-98af-552f9af0137d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:14.154466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2a054-0fb4-4426-98af-552f9af0137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:14.242650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8eb2a054-0fb4-4426-98af-552f9af0137d None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:14.243928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8eb2a054-0fb4-4426-98af-552f9af0137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:14.283727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:14.379756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 682/1367] 10.222.0.65 () {58 vars in 1059 bytes} [Mon Aug 29 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:14.388515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e72af-6a7f-4ef3-9ca1-101548c683ab None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:14.390011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a94e72af-6a7f-4ef3-9ca1-101548c683ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:14.560474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e72af-6a7f-4ef3-9ca1-101548c683ab None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:14.585618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:14.699564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:14.700177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:14.700509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:14.700746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:14.702591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:14.812692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:14.814868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510e31a-6e24-4519-8287-2604f1248476 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:14.841809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 686/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:15.276850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a672d-8450-4c64-8380-c4cbdfe39ccc None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:15.277892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a672d-8450-4c64-8380-c4cbdfe39ccc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:15.278419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a672d-8450-4c64-8380-c4cbdfe39ccc None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:15.280546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 683/1369] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:15.285859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f89f3-8364-40c5-a575-91eeba4802ad None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:15.286156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f89f3-8364-40c5-a575-91eeba4802ad None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:15.286445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f89f3-8364-40c5-a575-91eeba4802ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:15.324482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:15.364416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0d4f89f3-8364-40c5-a575-91eeba4802ad None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:15.366044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0d4f89f3-8364-40c5-a575-91eeba4802ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:15.479133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 687/1370] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:16.109814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269200b3-5d29-450a-887e-3fd24b208bea None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:16.110456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269200b3-5d29-450a-887e-3fd24b208bea None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:16.110892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269200b3-5d29-450a-887e-3fd24b208bea None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:16.113992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 684/1371] 10.222.0.65 () {54 vars in 964 bytes} [Mon Aug 29 21:19:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:16.118063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933e403-05c8-426b-9c63-7cb60564c87b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:16.118810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933e403-05c8-426b-9c63-7cb60564c87b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:16.118810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933e403-05c8-426b-9c63-7cb60564c87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:16.218678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c933e403-05c8-426b-9c63-7cb60564c87b None None] MFA Rules not processed for user `3fb180119c3a4f07933f7d24f4d22121`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:16.220317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c933e403-05c8-426b-9c63-7cb60564c87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:16.278132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:16.312941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 688/1372] 10.222.0.65 () {58 vars in 1061 bytes} [Mon Aug 29 21:19:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:16.321989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269200b3-5d29-450a-887e-3fd24b208bea None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:16.323936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-269200b3-5d29-450a-887e-3fd24b208bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:16.440756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269200b3-5d29-450a-887e-3fd24b208bea None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:16.484654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:16.523156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:16.524002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:16.524254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:16.524452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:16.526172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:16.661499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:16.663684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c004ac-2703-41f4-a4d5-43bd886fb0da service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:16.688346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 685/1373] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:19:17.069044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1c626-3de2-45e4-9bbf-d5f309e2f343 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:17.069342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1c626-3de2-45e4-9bbf-d5f309e2f343 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:17.069661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1c626-3de2-45e4-9bbf-d5f309e2f343 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:17.071086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 689/1374] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:17.077234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e4157-7f4d-4273-a973-c77c54655fb7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:17.078089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e4157-7f4d-4273-a973-c77c54655fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:17.078089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e4157-7f4d-4273-a973-c77c54655fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:17.155215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ca7e4157-7f4d-4273-a973-c77c54655fb7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:17.156505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ca7e4157-7f4d-4273-a973-c77c54655fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:17.178482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:17.273921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 686/1375] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:17.570815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1c626-3de2-45e4-9bbf-d5f309e2f343 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:17.576324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e3b1c626-3de2-45e4-9bbf-d5f309e2f343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:17.723760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1c626-3de2-45e4-9bbf-d5f309e2f343 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:17.781091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:17.845603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:17.846647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:17.847206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:17.848139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:17.849830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:17.942713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:17.944395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5f9ca-5c1c-4888-86ae-783b2cd3d6da service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:17.962745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 690/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:20.058219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aead3e2-4a4c-46c5-98b5-991707d08d5b None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:20.058785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aead3e2-4a4c-46c5-98b5-991707d08d5b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:20.058785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aead3e2-4a4c-46c5-98b5-991707d08d5b None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:20.060148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 687/1377] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:20.065367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608ea4-dc04-4d63-a3a3-479943f1d354 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:20.065775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608ea4-dc04-4d63-a3a3-479943f1d354 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:20.065921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608ea4-dc04-4d63-a3a3-479943f1d354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:20.143395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-99608ea4-dc04-4d63-a3a3-479943f1d354 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:20.144628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-99608ea4-dc04-4d63-a3a3-479943f1d354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:20.153115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:20.267991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 691/1378] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:20.563301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aead3e2-4a4c-46c5-98b5-991707d08d5b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:20.564885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7aead3e2-4a4c-46c5-98b5-991707d08d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:20.766675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aead3e2-4a4c-46c5-98b5-991707d08d5b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:20.771748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:20.945222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:20.945844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:20.946126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:20.946401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:20.949206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:21.046331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:21.048060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6623704-d0cf-400a-b499-6a22e886a3b7 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:21.070586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 688/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:23.278943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd88124-2622-4930-805c-44a21096afc5 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:23.279966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd88124-2622-4930-805c-44a21096afc5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:23.279966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd88124-2622-4930-805c-44a21096afc5 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:23.280949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 692/1380] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:23.286128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe7615-8618-4516-b33c-66b4d1711e23 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:23.286592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe7615-8618-4516-b33c-66b4d1711e23 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:23.286837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe7615-8618-4516-b33c-66b4d1711e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:23.299017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:23.359316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bcbe7615-8618-4516-b33c-66b4d1711e23 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:23.360546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bcbe7615-8618-4516-b33c-66b4d1711e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:23.468270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 689/1381] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:23.764719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd88124-2622-4930-805c-44a21096afc5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:23.766665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ccd88124-2622-4930-805c-44a21096afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:23.921007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:23.932175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd88124-2622-4930-805c-44a21096afc5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:24.063658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:24.063658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:24.063658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:24.063658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:24.064319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:24.168485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:24.170544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b2f5-2b8c-406b-bf22-7ec50dac01a9 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:24.183061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:24.191994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 693/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:26.421822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ef03f-ad2f-4fee-b9d8-0d16d2fefa19 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:26.422523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ef03f-ad2f-4fee-b9d8-0d16d2fefa19 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:26.422523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ef03f-ad2f-4fee-b9d8-0d16d2fefa19 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:26.423670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 690/1383] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:26.429357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159d07-ee27-4f93-927f-9aba1038e877 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:26.429926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159d07-ee27-4f93-927f-9aba1038e877 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:26.429926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159d07-ee27-4f93-927f-9aba1038e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:26.500222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ae159d07-ee27-4f93-927f-9aba1038e877 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:26.501479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ae159d07-ee27-4f93-927f-9aba1038e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:26.607203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 694/1384] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:26.911304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ef03f-ad2f-4fee-b9d8-0d16d2fefa19 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:26.912949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-565ef03f-ad2f-4fee-b9d8-0d16d2fefa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:27.017214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:27.093503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ef03f-ad2f-4fee-b9d8-0d16d2fefa19 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:27.234378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:27.235142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:27.235501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:27.235852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:27.238609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:27.295390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:27.343190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:27.344931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae636809-3886-4006-82a2-c98f52d3e9c8 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:27.364071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 691/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:29.351997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30edb1-9430-4eb1-bf71-8c640a2f3080 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:29.352724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30edb1-9430-4eb1-bf71-8c640a2f3080 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:29.352724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30edb1-9430-4eb1-bf71-8c640a2f3080 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:29.353903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 695/1386] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:29.360032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03942-dea1-489a-aaf7-f8c7fe0d1cf0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:29.360621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03942-dea1-489a-aaf7-f8c7fe0d1cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:29.360621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a03942-dea1-489a-aaf7-f8c7fe0d1cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:29.434586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-56a03942-dea1-489a-aaf7-f8c7fe0d1cf0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:29.435738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-56a03942-dea1-489a-aaf7-f8c7fe0d1cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:29.540158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 692/1387] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:29.833344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30edb1-9430-4eb1-bf71-8c640a2f3080 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:29.834896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2a30edb1-9430-4eb1-bf71-8c640a2f3080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:29.914654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:29.998303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30edb1-9430-4eb1-bf71-8c640a2f3080 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:30.136398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:30.137044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:30.137344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:30.137652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:30.139309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:30.180907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:30.234809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:30.236471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c9b0f-1511-49bc-80a5-57f03050a661 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:30.256592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 696/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:32.339767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4191c4d-3065-43e4-80f7-fa03756e0d7e None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:32.340438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4191c4d-3065-43e4-80f7-fa03756e0d7e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:32.340438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4191c4d-3065-43e4-80f7-fa03756e0d7e None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:32.341797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 693/1389] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:32.347267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:32.347715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:32.347824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:32.415947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:32.417663 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:32.424692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579ac5-3fea-4cd7-9e75-3f1d10e86fbb None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:32.424983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579ac5-3fea-4cd7-9e75-3f1d10e86fbb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:32.425262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e579ac5-3fea-4cd7-9e75-3f1d10e86fbb None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:32.426746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 694/1390] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:32.431418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d588f-517c-4770-8732-0bead9bda5e9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:32.431703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d588f-517c-4770-8732-0bead9bda5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:32.432020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d588f-517c-4770-8732-0bead9bda5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:32.468546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:32.505665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-391d588f-517c-4770-8732-0bead9bda5e9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:32.506899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-391d588f-517c-4770-8732-0bead9bda5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:32.521625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 697/1391] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:32.609390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 695/1392] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:19:32.816685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:32.817893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:32.832957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:32.913359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de422a26-444b-4041-89d6-944092f7af79 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:32.913652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de422a26-444b-4041-89d6-944092f7af79 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:32.914045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de422a26-444b-4041-89d6-944092f7af79 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:32.915497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 696/1393] 10.222.0.65 () {54 vars in 967 bytes} [Mon Aug 29 21:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:19:32.922061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:32.922432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:32.922783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:32.941845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50ca7f-7f73-4da7-ba76-f7a9cbae7d97 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:33.005544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] MFA Rules not processed for user `6ed41f2207eb4938b26f287a025f4bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:33.006880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:33.059344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:33.067251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:33.067956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:33.068236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:33.068507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:33.070115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:33.081001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:33.091051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 697/1394] 10.222.0.65 () {58 vars in 1064 bytes} [Mon Aug 29 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:19:33.098637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:33.106197 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09def750-ff32-4bd3-b41a-47f621a8c21f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:33.170670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:33.172715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda99b3a-19b9-4e03-b148-56372278ac38 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:33.192202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 698/1395] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:33.193244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:33.193870 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:33.194136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:33.194489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:33.196039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:33.319301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:33.320781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a18ddb-cf46-4ae8-b43c-caf7969f76ed service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:33.335688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:33.340919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 698/1396] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:19:34.326318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f68ca-160c-46a6-8718-5c99a9bb2544 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:34.326848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f68ca-160c-46a6-8718-5c99a9bb2544 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:34.326848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f68ca-160c-46a6-8718-5c99a9bb2544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:34.396643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-888f68ca-160c-46a6-8718-5c99a9bb2544 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:34.397831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-888f68ca-160c-46a6-8718-5c99a9bb2544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:34.489744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:34.493066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 699/1397] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:34.542892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babe246-a77e-4a34-8e58-4a10c6fc9c80 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:34.543365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babe246-a77e-4a34-8e58-4a10c6fc9c80 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:34.543467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2babe246-a77e-4a34-8e58-4a10c6fc9c80 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:34.544857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 699/1398] 10.222.0.65 () {54 vars in 943 bytes} [Mon Aug 29 21:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:34.551274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708850d7-f236-42b6-a0e4-749527e37eb9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:34.551568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708850d7-f236-42b6-a0e4-749527e37eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:34.551849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708850d7-f236-42b6-a0e4-749527e37eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:34.633824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-708850d7-f236-42b6-a0e4-749527e37eb9 None None] MFA Rules not processed for user `fb437e7c06504d0aa7891269ed02f9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:34.635028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-708850d7-f236-42b6-a0e4-749527e37eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:34.749210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 700/1399] 10.222.0.65 () {58 vars in 1040 bytes} [Mon Aug 29 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:19:34.758421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babe246-a77e-4a34-8e58-4a10c6fc9c80 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:34.759955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2babe246-a77e-4a34-8e58-4a10c6fc9c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:34.902999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2babe246-a77e-4a34-8e58-4a10c6fc9c80 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:34.991675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:35.012299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:35.012841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:35.013268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:35.013548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:35.015304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:35.107045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:35.108746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b1165-22ae-4fee-a8c7-2d04af339cfa service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:35.117027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 700/1400] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:19:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:19:35.285969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a107a1-a2d5-4e8f-bd14-6b22d7cad2b8 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:35.290245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a107a1-a2d5-4e8f-bd14-6b22d7cad2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:35.290245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a107a1-a2d5-4e8f-bd14-6b22d7cad2b8 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:35.290245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 701/1401] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:35.293143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72f14-a7ec-4943-acdf-9bbcef43c42b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:35.293440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72f14-a7ec-4943-acdf-9bbcef43c42b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:35.293721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72f14-a7ec-4943-acdf-9bbcef43c42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:35.365375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8cc72f14-a7ec-4943-acdf-9bbcef43c42b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:35.366590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8cc72f14-a7ec-4943-acdf-9bbcef43c42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:35.438540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:35.473447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 701/1402] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:35.767795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a107a1-a2d5-4e8f-bd14-6b22d7cad2b8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:35.769323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b8a107a1-a2d5-4e8f-bd14-6b22d7cad2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:35.840598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:35.976498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a107a1-a2d5-4e8f-bd14-6b22d7cad2b8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:36.120878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:36.121608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:36.121901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:36.122335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:36.124039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:36.160741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:36.242885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:36.244728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da30b3-c219-47d1-8285-2ae20719c9ed service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:36.267316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 702/1403] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:39.004151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5151d-4d28-45e6-9956-aad3e3762976 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:39.004752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5151d-4d28-45e6-9956-aad3e3762976 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:39.004752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5151d-4d28-45e6-9956-aad3e3762976 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:39.006333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 702/1404] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:39.012088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ce550-9882-436b-b959-d18ca62d5905 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:39.012534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ce550-9882-436b-b959-d18ca62d5905 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:39.012634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ce550-9882-436b-b959-d18ca62d5905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:39.095503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4d5ce550-9882-436b-b959-d18ca62d5905 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:39.096665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4d5ce550-9882-436b-b959-d18ca62d5905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:39.209255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:39.219335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 703/1405] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:19:39.511822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5151d-4d28-45e6-9956-aad3e3762976 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:39.514140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c9e5151d-4d28-45e6-9956-aad3e3762976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:39.685480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5151d-4d28-45e6-9956-aad3e3762976 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:39.757460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:39.829183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:39.829920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:39.830301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:39.830659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:39.832634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:39.939389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:39.941341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2498-b525-4ff9-a146-dab94fa29608 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:39.962519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 703/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:42.203708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0a22-4a8c-4445-9d59-4f3ed827aba6 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:42.204306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0a22-4a8c-4445-9d59-4f3ed827aba6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:42.204306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0a22-4a8c-4445-9d59-4f3ed827aba6 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:42.206067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 704/1407] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:42.211695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77450121-09ae-47c5-a57e-cd2c46922cb1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:42.212182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77450121-09ae-47c5-a57e-cd2c46922cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:42.212368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77450121-09ae-47c5-a57e-cd2c46922cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:42.296962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-77450121-09ae-47c5-a57e-cd2c46922cb1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:42.298495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-77450121-09ae-47c5-a57e-cd2c46922cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:42.313618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:42.425468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 704/1408] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:42.738969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0a22-4a8c-4445-9d59-4f3ed827aba6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:19:42.740664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d23f0a22-4a8c-4445-9d59-4f3ed827aba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:42.927754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0a22-4a8c-4445-9d59-4f3ed827aba6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:19:43.001936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:43.059407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:19:43.060345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:43.060801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:43.061260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:43.062942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:43.167457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:19:43.169370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c09c0-97cb-4e0b-a3b6-47fb71c5ad9b service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:19:43.189743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 705/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:19:52.663017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=38 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:53.187143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=64 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:59.180039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aec1-7d02-4424-b1ec-b4f8382a2db5 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:59.180601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aec1-7d02-4424-b1ec-b4f8382a2db5 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:59.180601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7aec1-7d02-4424-b1ec-b4f8382a2db5 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:59.182467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 705/1410] 10.222.0.65 () {54 vars in 918 bytes} [Mon Aug 29 21:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:59.187858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a84b6-8d3a-4a0a-866d-49a324c0e1a9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:59.188363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a84b6-8d3a-4a0a-866d-49a324c0e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:59.188460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a84b6-8d3a-4a0a-866d-49a324c0e1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:59.265335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-647a84b6-8d3a-4a0a-866d-49a324c0e1a9 None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:59.266601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-647a84b6-8d3a-4a0a-866d-49a324c0e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:59.377266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 706/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Mon Aug 29 21:19:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:19:59.392278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121138-10fa-4ca0-af5b-ec4c77781634 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:59.392565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121138-10fa-4ca0-af5b-ec4c77781634 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:59.392843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d121138-10fa-4ca0-af5b-ec4c77781634 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:59.394188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 706/1412] 10.222.0.65 () {54 vars in 918 bytes} [Mon Aug 29 21:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:19:59.398906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20b67f-5541-4eea-b034-8f500728d22d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:19:59.399192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20b67f-5541-4eea-b034-8f500728d22d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:19:59.399470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20b67f-5541-4eea-b034-8f500728d22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:19:59.474264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:19:59.477392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5c20b67f-5541-4eea-b034-8f500728d22d None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:19:59.478607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5c20b67f-5541-4eea-b034-8f500728d22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:19:59.596332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 707/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Mon Aug 29 21:19:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:02.015247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096012c-0f6e-4895-bb0e-aad222dd5769 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:02.015785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096012c-0f6e-4895-bb0e-aad222dd5769 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:02.015909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096012c-0f6e-4895-bb0e-aad222dd5769 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:02.017188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:02.022694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e44d8-3ce9-46cb-8102-c6ee68c829af None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:02.023175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e44d8-3ce9-46cb-8102-c6ee68c829af None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:02.023389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e44d8-3ce9-46cb-8102-c6ee68c829af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:02.106493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-aa7e44d8-3ce9-46cb-8102-c6ee68c829af None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:02.107780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-aa7e44d8-3ce9-46cb-8102-c6ee68c829af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:02.170277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:02.218249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:02.514369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096012c-0f6e-4895-bb0e-aad222dd5769 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:02.515905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5096012c-0f6e-4895-bb0e-aad222dd5769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:02.663546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096012c-0f6e-4895-bb0e-aad222dd5769 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:02.737861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:02.779529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:02.780129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:02.780416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:02.780694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:02.782304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:02.876717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:02.878408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ae544-6480-4e5f-bb3a-6b4d0a924a0e service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:02.897136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 708/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:03.801124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7b5e1-d51f-4825-b8c6-6d8220fd2862 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:03.801671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7b5e1-d51f-4825-b8c6-6d8220fd2862 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:03.801671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7b5e1-d51f-4825-b8c6-6d8220fd2862 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:03.803032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:03.808067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923232-2f8d-4919-bf26-5159cdf89631 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:03.808609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923232-2f8d-4919-bf26-5159cdf89631 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:03.808609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923232-2f8d-4919-bf26-5159cdf89631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:03.889533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8b923232-2f8d-4919-bf26-5159cdf89631 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:03.890738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8b923232-2f8d-4919-bf26-5159cdf89631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:03.917869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:03.989683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:04.139949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7b5e1-d51f-4825-b8c6-6d8220fd2862 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:04.141532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2ec7b5e1-d51f-4825-b8c6-6d8220fd2862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:04.292110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7b5e1-d51f-4825-b8c6-6d8220fd2862 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:04.332024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:04.406605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:04.407143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:04.407425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:04.407699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:04.409363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:04.499591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:04.501260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb959af7-009c-4abf-9aa8-1fdd202c9115 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:04.520146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 710/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:05.418410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c2824-e688-40cd-9637-4c19fa7b1bc5 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.418945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c2824-e688-40cd-9637-4c19fa7b1bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.418945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c2824-e688-40cd-9637-4c19fa7b1bc5 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.420250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:05.425407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48ffca-915e-45e8-b69d-674fbe23047f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:05.425874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48ffca-915e-45e8-b69d-674fbe23047f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:05.425979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48ffca-915e-45e8-b69d-674fbe23047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:05.483147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:05.506440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cf48ffca-915e-45e8-b69d-674fbe23047f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:05.507712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cf48ffca-915e-45e8-b69d-674fbe23047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:05.606568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:05.756758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c2824-e688-40cd-9637-4c19fa7b1bc5 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:05.758268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6a9c2824-e688-40cd-9637-4c19fa7b1bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:06.025200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c2824-e688-40cd-9637-4c19fa7b1bc5 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:06.037886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:06.140764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:06.141190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:06.141471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:06.141744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:06.143485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:06.248426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:06.250035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff2cb7-89a9-4297-90cc-a8a0d2c64360 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:06.268263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 711/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:07.132674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d29c42-b034-4d39-a823-356c562e63d8 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:07.133177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d29c42-b034-4d39-a823-356c562e63d8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:07.133302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d29c42-b034-4d39-a823-356c562e63d8 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:07.134621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:07.140196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a129c4-5eea-44f9-8897-f2298b3bcbe9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:07.140639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a129c4-5eea-44f9-8897-f2298b3bcbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:07.140771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a129c4-5eea-44f9-8897-f2298b3bcbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:07.160816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:07.221659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e7a129c4-5eea-44f9-8897-f2298b3bcbe9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:07.222899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e7a129c4-5eea-44f9-8897-f2298b3bcbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:07.347354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:07.505935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d29c42-b034-4d39-a823-356c562e63d8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:07.507404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-93d29c42-b034-4d39-a823-356c562e63d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:07.630317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:07.665684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d29c42-b034-4d39-a823-356c562e63d8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:07.781875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:07.782388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:07.782668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:07.782938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:07.784589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:07.870348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:07.883551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:07.885752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e237ff-0817-43db-aaf9-3dbdab74a607 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:07.909063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 713/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:08.779144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f6a57-148c-4ccc-aed8-e3e6e8f16f07 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:08.779729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f6a57-148c-4ccc-aed8-e3e6e8f16f07 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:08.779729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f6a57-148c-4ccc-aed8-e3e6e8f16f07 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:08.780997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:08.785924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8e3cc-80f4-45f0-a97e-cc850331bc77 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:08.786401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8e3cc-80f4-45f0-a97e-cc850331bc77 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:08.786521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8e3cc-80f4-45f0-a97e-cc850331bc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:08.865815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d2c8e3cc-80f4-45f0-a97e-cc850331bc77 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:08.867030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d2c8e3cc-80f4-45f0-a97e-cc850331bc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:08.966324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:20:09.117105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f6a57-148c-4ccc-aed8-e3e6e8f16f07 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:09.118642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-312f6a57-148c-4ccc-aed8-e3e6e8f16f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:09.207310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.268792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f6a57-148c-4ccc-aed8-e3e6e8f16f07 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:09.393716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:09.394257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:09.394528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:09.394800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:09.396511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:09.455479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:09.502801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:09.504448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7cc07-37c3-4fb2-a11a-54799746486c service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:09.522771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 714/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:10.391827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c43f1f-4d44-4359-95da-408e8d1c489c None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:10.392381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c43f1f-4d44-4359-95da-408e8d1c489c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:10.392381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c43f1f-4d44-4359-95da-408e8d1c489c None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:10.393670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:10.398813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f15c0-3ae1-4517-a205-a2efb06a7d40 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:10.399290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f15c0-3ae1-4517-a205-a2efb06a7d40 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:10.399398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f15c0-3ae1-4517-a205-a2efb06a7d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:10.476648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-101f15c0-3ae1-4517-a205-a2efb06a7d40 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:10.477870 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-101f15c0-3ae1-4517-a205-a2efb06a7d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:10.576399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:10.726850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c43f1f-4d44-4359-95da-408e8d1c489c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:10.728338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-12c43f1f-4d44-4359-95da-408e8d1c489c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:10.777736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:10.879292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c43f1f-4d44-4359-95da-408e8d1c489c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:10.993005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:10.993558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:10.993835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:10.994125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:10.995717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:11.002975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:11.086556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:11.088845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af996ecb-01fa-4f47-a479-14a0e69cf211 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:11.107201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 716/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:11.972554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476febcd-08fa-4da1-b319-4bd5b3ac36cc None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:11.973129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476febcd-08fa-4da1-b319-4bd5b3ac36cc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:11.973129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476febcd-08fa-4da1-b319-4bd5b3ac36cc None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:11.974520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:11.980433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf53f8-4ece-4245-b602-6c43ecc8e24c None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:11.980878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf53f8-4ece-4245-b602-6c43ecc8e24c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:11.980986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf53f8-4ece-4245-b602-6c43ecc8e24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:12.060226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-59cf53f8-4ece-4245-b602-6c43ecc8e24c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:12.061443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-59cf53f8-4ece-4245-b602-6c43ecc8e24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:12.134001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:12.165889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:12.317864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476febcd-08fa-4da1-b319-4bd5b3ac36cc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:12.319628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-476febcd-08fa-4da1-b319-4bd5b3ac36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:12.339139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:12.469928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476febcd-08fa-4da1-b319-4bd5b3ac36cc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:12.557771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:12.583091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:12.583699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:12.584048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:12.584288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:12.585867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:12.675408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:12.677046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed4876-3c3c-4674-892a-69dba0afd04a service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:12.695374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 717/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:13.597766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79d82-ef9a-414c-8cd8-324201d20eac None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:13.598367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79d82-ef9a-414c-8cd8-324201d20eac None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:13.598367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79d82-ef9a-414c-8cd8-324201d20eac None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:13.599735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:13.605182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b653a-ed2e-414f-ae88-c3f375e84216 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:13.605651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b653a-ed2e-414f-ae88-c3f375e84216 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:13.605769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b653a-ed2e-414f-ae88-c3f375e84216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:13.682095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-033b653a-ed2e-414f-ae88-c3f375e84216 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:13.683305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-033b653a-ed2e-414f-ae88-c3f375e84216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:13.726817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:13.790401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:20:13.942738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e79d82-ef9a-414c-8cd8-324201d20eac None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:13.944225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-05e79d82-ef9a-414c-8cd8-324201d20eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:14.099093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e79d82-ef9a-414c-8cd8-324201d20eac None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:14.160907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:14.221217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:14.221780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:14.222085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:14.222360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:14.224001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:14.316857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:14.319476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9853f-30bd-429b-9697-7c222ee24b95 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:14.340634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 719/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:15.261188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae1c-c93c-4bc8-980c-b82488f45c60 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:15.261909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae1c-c93c-4bc8-980c-b82488f45c60 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:15.261909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae1c-c93c-4bc8-980c-b82488f45c60 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:15.263083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:15.268167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20ac84-e405-491c-ac32-93422a79fddf None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:15.268586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20ac84-e405-491c-ac32-93422a79fddf None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:15.268723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20ac84-e405-491c-ac32-93422a79fddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:15.346186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7e20ac84-e405-491c-ac32-93422a79fddf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:15.347460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7e20ac84-e405-491c-ac32-93422a79fddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:15.349621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:15.450073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:15.600345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae1c-c93c-4bc8-980c-b82488f45c60 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:15.602504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9be1ae1c-c93c-4bc8-980c-b82488f45c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:15.756979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae1c-c93c-4bc8-980c-b82488f45c60 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:15.787645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:15.879543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:15.880200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:15.880480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:15.880754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:15.882352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:15.974133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:15.975720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7807998-c5c0-4ac6-81ad-d319b84a1a22 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:15.995097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 720/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:17.885026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fc4d3-a877-45e3-bbc7-9423d6beb5c1 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.888019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fc4d3-a877-45e3-bbc7-9423d6beb5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.888019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00fc4d3-a877-45e3-bbc7-9423d6beb5c1 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.888019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:17.893277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d0847-d49a-4d5e-ba78-7cf6eac78741 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:17.893735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d0847-d49a-4d5e-ba78-7cf6eac78741 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:17.893826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d0847-d49a-4d5e-ba78-7cf6eac78741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:17.936068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:17.961705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-083d0847-d49a-4d5e-ba78-7cf6eac78741 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:17.962932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-083d0847-d49a-4d5e-ba78-7cf6eac78741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:18.060772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:18.357086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fc4d3-a877-45e3-bbc7-9423d6beb5c1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:18.358658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c00fc4d3-a877-45e3-bbc7-9423d6beb5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:18.465647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00fc4d3-a877-45e3-bbc7-9423d6beb5c1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:18.497862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:18.538722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:18.539330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:18.539627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:18.539901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:18.541540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:18.655521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:18.657035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1322c5-84e4-42ff-8194-27990105dd35 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:18.676455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 722/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:20:20.772427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf6a1-883e-4308-b65c-fdc23ee70c40 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.772991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf6a1-883e-4308-b65c-fdc23ee70c40 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.772991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdf6a1-883e-4308-b65c-fdc23ee70c40 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.774331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:20.780218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b412-1473-44d3-9dd3-8abf2b40ba85 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:20.780699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b412-1473-44d3-9dd3-8abf2b40ba85 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:20.780804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b412-1473-44d3-9dd3-8abf2b40ba85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:20.852189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d9e4b412-1473-44d3-9dd3-8abf2b40ba85 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:20.854277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d9e4b412-1473-44d3-9dd3-8abf2b40ba85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:20.867808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:20.952722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:21.240856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf6a1-883e-4308-b65c-fdc23ee70c40 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:21.242441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-58cdf6a1-883e-4308-b65c-fdc23ee70c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:21.345983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdf6a1-883e-4308-b65c-fdc23ee70c40 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:21.358334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:21.419344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:21.419864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:21.420180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:21.420452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:21.422062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:21.531201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:21.532671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d1bb2-cd21-4543-a2db-c811f493dd30 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:21.550938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 723/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:23.868151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ae890-622e-4080-9109-252632110f84 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.868698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ae890-622e-4080-9109-252632110f84 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.868698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ae890-622e-4080-9109-252632110f84 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.870686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:23.876023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363575e0-5675-4247-a2d9-86b1060f4d78 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:23.876445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363575e0-5675-4247-a2d9-86b1060f4d78 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:23.880909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363575e0-5675-4247-a2d9-86b1060f4d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:23.938355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:23.957238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-363575e0-5675-4247-a2d9-86b1060f4d78 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:23.958448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-363575e0-5675-4247-a2d9-86b1060f4d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:24.070852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:24.361218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ae890-622e-4080-9109-252632110f84 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:24.362781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d19ae890-622e-4080-9109-252632110f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:24.469533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ae890-622e-4080-9109-252632110f84 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:24.511371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:24.544620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:24.545227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:24.545503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:24.545772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:24.547372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:24.658079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:24.659532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5284fd-d18b-43f3-a944-95bccf057a48 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:24.682386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 725/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:26.743047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204ee2-f780-4bce-9f5e-7755bd2b4d89 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:26.743612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204ee2-f780-4bce-9f5e-7755bd2b4d89 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:26.743612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83204ee2-f780-4bce-9f5e-7755bd2b4d89 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:26.745517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:26.751209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625939b-bc0e-4ed7-af4d-68f9b8edd57a None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:26.751654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625939b-bc0e-4ed7-af4d-68f9b8edd57a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:26.751771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625939b-bc0e-4ed7-af4d-68f9b8edd57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:26.834699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0625939b-bc0e-4ed7-af4d-68f9b8edd57a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:26.836010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0625939b-bc0e-4ed7-af4d-68f9b8edd57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:26.857769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:26.939436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:27.226941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204ee2-f780-4bce-9f5e-7755bd2b4d89 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:27.228604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-83204ee2-f780-4bce-9f5e-7755bd2b4d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:27.351818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83204ee2-f780-4bce-9f5e-7755bd2b4d89 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:27.386835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:27.474355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:27.474968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:27.475247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:27.475520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:27.477132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:27.586476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:27.591766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57266e1f-4d58-4f3f-b22d-85e1c8418e63 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:27.608541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 726/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:29.616793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31accc2-baea-42f0-8ba6-242c8138dc1e None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.617336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31accc2-baea-42f0-8ba6-242c8138dc1e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.617336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31accc2-baea-42f0-8ba6-242c8138dc1e None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.618715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:29.624203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859538f8-6aba-478e-bc90-aa6348939fe3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:29.624618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859538f8-6aba-478e-bc90-aa6348939fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:29.624759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859538f8-6aba-478e-bc90-aa6348939fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:29.691368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:29.694265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-859538f8-6aba-478e-bc90-aa6348939fe3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:29.695486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-859538f8-6aba-478e-bc90-aa6348939fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:29.795426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:30.086005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31accc2-baea-42f0-8ba6-242c8138dc1e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:30.087672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a31accc2-baea-42f0-8ba6-242c8138dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:30.195547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31accc2-baea-42f0-8ba6-242c8138dc1e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:30.248954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:30.276781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:30.277399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:30.277684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:30.277992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:30.279635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:30.394676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:30.396081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808dc0b-9ed1-4231-9f58-42119ad533f8 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:30.415164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 728/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:32.453795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8ec6-2faf-4140-8f7a-179da89e248c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.454328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8ec6-2faf-4140-8f7a-179da89e248c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.454328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8ec6-2faf-4140-8f7a-179da89e248c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.455610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:32.461286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9713ac4-f018-411c-9c45-2901047433d4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:32.461741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9713ac4-f018-411c-9c45-2901047433d4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:32.461836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9713ac4-f018-411c-9c45-2901047433d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:32.532360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d9713ac4-f018-411c-9c45-2901047433d4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:32.533568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d9713ac4-f018-411c-9c45-2901047433d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:32.562493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:32.636232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:32.923209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8ec6-2faf-4140-8f7a-179da89e248c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:32.924832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-17bc8ec6-2faf-4140-8f7a-179da89e248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:33.045260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8ec6-2faf-4140-8f7a-179da89e248c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:33.075534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:33.124821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:33.125444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:33.125720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:33.125990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:33.127574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:33.245519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:33.246868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbfc9d-39ba-4a28-92bc-a336323826fb service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:33.265432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 729/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:20:35.235453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84014e-c8b9-45dd-af5f-e755df28a75f None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.236074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84014e-c8b9-45dd-af5f-e755df28a75f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.236074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84014e-c8b9-45dd-af5f-e755df28a75f None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.237477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:35.243411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76fa55-0aec-4c88-ab47-6a3d55b663e3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.253045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76fa55-0aec-4c88-ab47-6a3d55b663e3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.253045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76fa55-0aec-4c88-ab47-6a3d55b663e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.327396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1b76fa55-0aec-4c88-ab47-6a3d55b663e3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:35.328702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1b76fa55-0aec-4c88-ab47-6a3d55b663e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:35.338582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:35.448856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:35.737065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84014e-c8b9-45dd-af5f-e755df28a75f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:35.738644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-eb84014e-c8b9-45dd-af5f-e755df28a75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:35.864052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84014e-c8b9-45dd-af5f-e755df28a75f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:35.927263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:35.948606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:35.949435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:35.949583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:35.949850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:35.951948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:36.084142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:36.085635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced35fdc-d8fd-495a-829f-d09d1db5bc18 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:36.110575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 731/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:38.149053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c0632-3628-493e-8df4-6fa308c0d33a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.149622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c0632-3628-493e-8df4-6fa308c0d33a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.149622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c0632-3628-493e-8df4-6fa308c0d33a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.151624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:38.157651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d424e-839a-4a3b-9735-a04b7c4c05f6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.158468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d424e-839a-4a3b-9735-a04b7c4c05f6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.158468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d424e-839a-4a3b-9735-a04b7c4c05f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.233656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-501d424e-839a-4a3b-9735-a04b7c4c05f6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:38.234907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-501d424e-839a-4a3b-9735-a04b7c4c05f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:38.287683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.343632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:38.639765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c0632-3628-493e-8df4-6fa308c0d33a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:38.641332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-669c0632-3628-493e-8df4-6fa308c0d33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:38.748478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c0632-3628-493e-8df4-6fa308c0d33a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:38.786769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:38.826187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:38.826862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:38.827104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:38.827382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:38.829234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:38.958841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:38.960455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762f069-dba6-4ff7-8820-8240a63bbf4b service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:38.981619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 732/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:40.975160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f21fc-fd5a-4f28-821a-6f0e7bb8a585 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.975753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f21fc-fd5a-4f28-821a-6f0e7bb8a585 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.975753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f21fc-fd5a-4f28-821a-6f0e7bb8a585 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:40.977150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:40.983254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b63867-09eb-4c17-bfdc-1e98e7dfa026 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:40.983777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b63867-09eb-4c17-bfdc-1e98e7dfa026 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:40.983880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b63867-09eb-4c17-bfdc-1e98e7dfa026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.055253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-92b63867-09eb-4c17-bfdc-1e98e7dfa026 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:41.057039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-92b63867-09eb-4c17-bfdc-1e98e7dfa026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:41.079091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.163697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:41.450681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f21fc-fd5a-4f28-821a-6f0e7bb8a585 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:41.452317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fe3f21fc-fd5a-4f28-821a-6f0e7bb8a585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:41.569926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f21fc-fd5a-4f28-821a-6f0e7bb8a585 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:41.643451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:41.650345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:41.650963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:41.651243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:41.651520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:41.653252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:41.779335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:41.780760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4586b82-ab7f-4c8c-a088-9ecf86a3a4aa service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:41.800784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 734/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:43.777206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f354-0c96-4039-b06d-2ff142ecc0f2 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.778036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f354-0c96-4039-b06d-2ff142ecc0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.778036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6f354-0c96-4039-b06d-2ff142ecc0f2 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.779133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:20:43.784998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7446bf7-615d-45a4-bec7-6bc14556e425 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:43.785506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7446bf7-615d-45a4-bec7-6bc14556e425 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:43.785622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7446bf7-615d-45a4-bec7-6bc14556e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:43.866390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c7446bf7-615d-45a4-bec7-6bc14556e425 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:20:43.867697 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c7446bf7-615d-45a4-bec7-6bc14556e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:43.922481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:43.975128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:20:44.257186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f354-0c96-4039-b06d-2ff142ecc0f2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:20:44.258733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c2b6f354-0c96-4039-b06d-2ff142ecc0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:44.370699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6f354-0c96-4039-b06d-2ff142ecc0f2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:20:44.428005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:44.460553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:20:44.461198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:20:44.461474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:20:44.461881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:20:44.463562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:20:44.590777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:20:44.592285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c9543-7359-4ee4-bcb3-31501a1aff12 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:20:44.614210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 735/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:20:53.973527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=22 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:20:54.611538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=66 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:04.747629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e4384-c0a0-400a-93b8-229c92e06d39 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.748264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e4384-c0a0-400a-93b8-229c92e06d39 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.748264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e4384-c0a0-400a-93b8-229c92e06d39 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.749610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 736/1471] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:04.755217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a421c-6dd8-4dc2-84d0-c0be1b038b38 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:04.755728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a421c-6dd8-4dc2-84d0-c0be1b038b38 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:04.755825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a421c-6dd8-4dc2-84d0-c0be1b038b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:04.833762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-da8a421c-6dd8-4dc2-84d0-c0be1b038b38 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:04.835096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-da8a421c-6dd8-4dc2-84d0-c0be1b038b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:04.942557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 736/1472] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:05.237379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b11bd-43ee-4767-b211-9e0d7a5dac4f None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.237666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b11bd-43ee-4767-b211-9e0d7a5dac4f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.238006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b11bd-43ee-4767-b211-9e0d7a5dac4f None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.239387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 737/1473] 10.222.0.65 () {54 vars in 967 bytes} [Mon Aug 29 21:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:21:05.244770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e413ae-a6e5-490b-806b-7e14e17c01b7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.245056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e413ae-a6e5-490b-806b-7e14e17c01b7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.245339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e413ae-a6e5-490b-806b-7e14e17c01b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.326485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-15e413ae-a6e5-490b-806b-7e14e17c01b7 None None] MFA Rules not processed for user `6ed41f2207eb4938b26f287a025f4bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:05.327726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-15e413ae-a6e5-490b-806b-7e14e17c01b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:05.354235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:05.413353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 737/1474] 10.222.0.65 () {58 vars in 1064 bytes} [Mon Aug 29 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:05.421539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b11bd-43ee-4767-b211-9e0d7a5dac4f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:05.423053 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-315b11bd-43ee-4767-b211-9e0d7a5dac4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:05.529349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b11bd-43ee-4767-b211-9e0d7a5dac4f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:05.603843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:05.604490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:05.604767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:05.605036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:05.606671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:05.661877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:05.732411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:05.733925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e585f9-dcf6-4b45-8ce6-3b2aaa67428a service glance] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:05.756701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 738/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:21:07.804134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f29365-70c9-4d93-b776-bd4e0189e52a None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.804753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f29365-70c9-4d93-b776-bd4e0189e52a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.804753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f29365-70c9-4d93-b776-bd4e0189e52a None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.806691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 738/1476] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:07.812145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532b991-f026-42e1-8c69-44027912b933 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:07.812650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532b991-f026-42e1-8c69-44027912b933 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:07.812748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532b991-f026-42e1-8c69-44027912b933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:07.888912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d532b991-f026-42e1-8c69-44027912b933 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:07.890140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d532b991-f026-42e1-8c69-44027912b933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:07.989804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 739/1477] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:08.266274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f29365-70c9-4d93-b776-bd4e0189e52a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:08.267820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-14f29365-70c9-4d93-b776-bd4e0189e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:08.413754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f29365-70c9-4d93-b776-bd4e0189e52a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:08.465724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:08.511449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:08.512088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:08.512366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:08.512633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:08.513270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:08.515022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:21:08.515022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:21:08.515022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:21:08.516175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d79a9-c478-4dc8-88f1-3a3349987611 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:08.535015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 739/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Mon Aug 29 21:21:08 2022] GET /identity/v3/projects => generated 2553 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:21:10.514021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b458ede-7ecf-4617-b889-82b4d47b4f6c None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:10.514640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b458ede-7ecf-4617-b889-82b4d47b4f6c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:10.514640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b458ede-7ecf-4617-b889-82b4d47b4f6c None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:10.515842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 740/1479] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:10.521283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2c1f5-5426-4f06-b653-9461db69bd8f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:10.521741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2c1f5-5426-4f06-b653-9461db69bd8f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:10.521847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2c1f5-5426-4f06-b653-9461db69bd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:10.590039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6bb2c1f5-5426-4f06-b653-9461db69bd8f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:10.591269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6bb2c1f5-5426-4f06-b653-9461db69bd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:10.689828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 740/1480] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:10.986341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b458ede-7ecf-4617-b889-82b4d47b4f6c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:10.987923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3b458ede-7ecf-4617-b889-82b4d47b4f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:10.998429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:11.141595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b458ede-7ecf-4617-b889-82b4d47b4f6c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:11.228200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:11.263140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:11.263759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:11.264079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:11.264358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:11.265972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:11.361292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:11.362908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90770f7e-9a56-4397-a573-7bb2773be458 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:11.382725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 741/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:21:13.512273 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b14f-f0fa-4add-b233-b078d1ec71bc None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.512845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b14f-f0fa-4add-b233-b078d1ec71bc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.512845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b14f-f0fa-4add-b233-b078d1ec71bc None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.514145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 741/1482] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:13.519582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba375-3b13-4658-9b19-12a98f683da2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:13.520076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba375-3b13-4658-9b19-12a98f683da2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:13.520180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba375-3b13-4658-9b19-12a98f683da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:13.589880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-bcdba375-3b13-4658-9b19-12a98f683da2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:13.591082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bcdba375-3b13-4658-9b19-12a98f683da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:13.640692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:13.692532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 742/1483] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:13.987464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b14f-f0fa-4add-b233-b078d1ec71bc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:13.989093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7ff1b14f-f0fa-4add-b233-b078d1ec71bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:14.015275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:14.111171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b14f-f0fa-4add-b233-b078d1ec71bc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:14.200979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:14.201905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:14.202222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:14.202499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:14.204193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:14.297881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:14.338851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:14.340355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1869c8f-0a95-41d3-93f7-df6ad83ff4d6 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:14.359945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 742/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:21:16.426870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fb7d-3cd8-4aa1-a3e3-fd408d35e83e None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.427622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fb7d-3cd8-4aa1-a3e3-fd408d35e83e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.427622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fb7d-3cd8-4aa1-a3e3-fd408d35e83e None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.428847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 743/1485] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:16.435034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723dc8-1b69-4c11-acd3-816129d370ef None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:16.435542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723dc8-1b69-4c11-acd3-816129d370ef None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:16.435713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723dc8-1b69-4c11-acd3-816129d370ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:16.511395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e2723dc8-1b69-4c11-acd3-816129d370ef None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:16.512840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e2723dc8-1b69-4c11-acd3-816129d370ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:16.615436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 743/1486] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:16.921057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fb7d-3cd8-4aa1-a3e3-fd408d35e83e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:16.922676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4fa8fb7d-3cd8-4aa1-a3e3-fd408d35e83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:17.051577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fb7d-3cd8-4aa1-a3e3-fd408d35e83e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:17.138544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:17.142414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:17.142414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:17.142414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:17.142414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:17.143128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:17.294891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:17.296390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be57770-e71b-4382-9268-032f12ea2d47 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:17.316300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 744/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:21:19.541765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60025-97c5-4633-b0b5-7181b4443d73 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.542684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60025-97c5-4633-b0b5-7181b4443d73 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.542684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60025-97c5-4633-b0b5-7181b4443d73 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.545027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 744/1488] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:19.552068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc0bb-76a6-4936-8a76-24ce78d444b6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:19.552660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc0bb-76a6-4936-8a76-24ce78d444b6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:19.553032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc0bb-76a6-4936-8a76-24ce78d444b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:19.650968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-efdfc0bb-76a6-4936-8a76-24ce78d444b6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:19.652234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-efdfc0bb-76a6-4936-8a76-24ce78d444b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:19.708900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:19.762136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 745/1489] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:21:20.059802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60025-97c5-4633-b0b5-7181b4443d73 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:20.061986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8ce60025-97c5-4633-b0b5-7181b4443d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:20.101061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.172880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60025-97c5-4633-b0b5-7181b4443d73 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:20.254131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:20.254745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:20.255028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:20.255305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:20.256968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:20.345348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:20.382731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:20.384726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64847e57-862b-4d8f-a7ef-6cf0e5b500dc service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:20.405445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 745/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:21:23.011418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b89850-5c20-4a2b-8963-38daf44ff1cd None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.012241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b89850-5c20-4a2b-8963-38daf44ff1cd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.012241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b89850-5c20-4a2b-8963-38daf44ff1cd None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.013533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 746/1491] 10.222.0.65 () {54 vars in 927 bytes} [Mon Aug 29 21:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:21:23.019918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7502920-518d-4699-a153-f563a13623c3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.020516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7502920-518d-4699-a153-f563a13623c3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.020516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7502920-518d-4699-a153-f563a13623c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.104224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a7502920-518d-4699-a153-f563a13623c3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:21:23.105463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a7502920-518d-4699-a153-f563a13623c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:23.214039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 746/1492] 10.222.0.65 () {58 vars in 1024 bytes} [Mon Aug 29 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:21:23.508912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b89850-5c20-4a2b-8963-38daf44ff1cd None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:21:23.511047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c3b89850-5c20-4a2b-8963-38daf44ff1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:23.647144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b89850-5c20-4a2b-8963-38daf44ff1cd None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:21:23.781041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:21:23.781947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:21:23.782396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:21:23.782833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:21:23.785995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:21:23.805007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:23.938834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:21:23.940368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880216cb-8b3f-48f3-b715-5a2748052a76 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:21:23.965322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 747/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:21:33.210902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=82 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:21:33.961967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=50 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:51.526213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:51.526881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:51.526881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:51.606390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:51.607608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:51.707618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 747/1494] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:22:51.716230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:51.716890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:51.716890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:51.800593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:51.801797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:51.925778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 748/1495] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:22:51.931695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:51.933355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:51.945707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5bab5-c8ae-4cb5-98d2-bd7aeb25ff22 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:52.012643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:52.053709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:52.054309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:52.054586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:52.054859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:52.055512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:22:52.055814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:52.056879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:52.056879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:22:52.056879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:22:52.057770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:52.070485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 748/1496] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:22:52.079880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:52.090327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cd1e6-faaf-4c9a-b3c0-40d6f6c14dab None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:52.156647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:52.194272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:52.194928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:52.195712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:52.195712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:52.196672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:52.198747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:52.198747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:22:52.198747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:22:52.200033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:52.344246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 749/1497] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:22:52 2022] POST /identity/v3/projects => generated 377 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:52.349010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:52.351722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:52.392193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177da82e-0fe8-43d3-8a36-bd56f58c2998 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:52.504610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:52.505287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:52.505597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:52.505900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:52.506569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:52.508189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:52.508189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:22:52.508189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:22:52.509542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:52.614660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 749/1498] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:22:52 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:22:52.618861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:52.620459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:52.686416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bc2cd-a2f0-4006-bc5c-f13ffb52241e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:52.775678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:52.786877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:22:52.792036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:22:52.792601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:52.806744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 750/1499] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:22:52.811416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:52.813014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:52.825026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98711a-9f0f-46d3-a2a2-0c167b59f280 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:52.854604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:52.937298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:52.937834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:52.938116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:52.938385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:52.939029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:52.940086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:52.940086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:22:52.940086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:22:52.941015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deffa9-b033-40ec-903f-17e7167c69a6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:52.955149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 750/1500] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:22:52.959445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:52.962098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:53.000373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046b85e-e000-4908-929d-d3aa0630fe25 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:53.110401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:53.111128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.111405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.111677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] PATH_INFO: `/v3/projects/f1f82567d6694e83997d15dfc9d9f238/users/608f2cece3e84294a60ed5ba6254332b/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.137830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f82567d6694e83997d15dfc9d9f238, user_id=608f2cece3e84294a60ed5ba6254332b, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:53.138889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:53.138889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:22:53.138889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:22:53.141023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae3c11-1429-472d-9343-f6de78676844 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:53.255222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 751/1501] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:22:52 2022] PUT /identity/v3/projects/f1f82567d6694e83997d15dfc9d9f238/users/608f2cece3e84294a60ed5ba6254332b/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:22:53.268979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ebf42-3f02-4982-a998-c63e9b443e9c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.269268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ebf42-3f02-4982-a998-c63e9b443e9c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.269550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ebf42-3f02-4982-a998-c63e9b443e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.331509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-639ebf42-3f02-4982-a998-c63e9b443e9c None None] MFA Rules not processed for user `608f2cece3e84294a60ed5ba6254332b`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:53.332802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-639ebf42-3f02-4982-a998-c63e9b443e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:53.415372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 751/1502] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:22:53.428858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f987-b6be-4637-97a0-935c52ca7747 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.429145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f987-b6be-4637-97a0-935c52ca7747 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.429421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f987-b6be-4637-97a0-935c52ca7747 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.430795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 752/1503] 10.222.0.65 () {54 vars in 964 bytes} [Mon Aug 29 21:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:22:53.435849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efc2b2-7d78-4985-a3de-58fb29b2dca6 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.436167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efc2b2-7d78-4985-a3de-58fb29b2dca6 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.436447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efc2b2-7d78-4985-a3de-58fb29b2dca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.471495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:53.525798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-73efc2b2-7d78-4985-a3de-58fb29b2dca6 None None] MFA Rules not processed for user `3fb180119c3a4f07933f7d24f4d22121`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:53.526971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-73efc2b2-7d78-4985-a3de-58fb29b2dca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:53.608811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 752/1504] 10.222.0.65 () {58 vars in 1061 bytes} [Mon Aug 29 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:22:53.617013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f987-b6be-4637-97a0-935c52ca7747 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:53.618652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4239f987-b6be-4637-97a0-935c52ca7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:53.673635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:53.731848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f987-b6be-4637-97a0-935c52ca7747 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:53.812299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:53.812954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:53.813149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:53.813381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:53.815057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:54.045698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:54.047169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:54.051636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:54.067167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 753/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:22:54.120848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b12a99-f3f9-41e5-a746-c633f14d3080 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:54.121416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b12a99-f3f9-41e5-a746-c633f14d3080 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:54.121416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b12a99-f3f9-41e5-a746-c633f14d3080 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:54.122849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 753/1506] 10.222.0.65 () {52 vars in 830 bytes} [Mon Aug 29 21:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:22:54.128099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:54.129504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:54.246042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55085a-3f89-4017-989a-f5ebcc77be83 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:54.341712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:54.342358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:54.342639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:54.342917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] PATH_INFO: `/v3/users/608f2cece3e84294a60ed5ba6254332b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:54.357794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=608f2cece3e84294a60ed5ba6254332b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:54.358856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:54.358856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:22:54.358856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:22:54.360115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:54.370897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:54.581439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608f2cece3e84294a60ed5ba6254332b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:54.581884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608f2cece3e84294a60ed5ba6254332b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:54.609045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608f2cece3e84294a60ed5ba6254332b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:54.609548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608f2cece3e84294a60ed5ba6254332b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:54.626198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608f2cece3e84294a60ed5ba6254332b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:54.686400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 754/1507] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:22:54 2022] DELETE /identity/v3/users/608f2cece3e84294a60ed5ba6254332b => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:22:54.691298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:54.691616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:54.691979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:54.751802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:22:54.753080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:54.836980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:54.850669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 754/1508] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:22:54.860489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:54.861708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:54.968347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfad67-135a-4730-b79a-c38864a58d67 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:55.024686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:22:55.043768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:55.044609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:55.044892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:55.045168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:55.046794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:55.164993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:55.166445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c1dcc-3f61-42f7-9c05-38297a5910df service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:55.185633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 755/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:22:55.647846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:22:55.649325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:22:55.661522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a9c4f-58ba-44c2-aed0-2fed08e20f4e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:22:55.770893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:22:55.771541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:22:55.771824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:22:55.772138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] PATH_INFO: `/v3/projects/f1f82567d6694e83997d15dfc9d9f238` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:22:55.782791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f82567d6694e83997d15dfc9d9f238)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:22:55.783856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:22:55.783856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:22:55.783856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:22:55.785075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:22:55.920556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f82567d6694e83997d15dfc9d9f238', 'request_id': 'req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:55.926881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f82567d6694e83997d15dfc9d9f238', 'request_id': 'req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:56.028066 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f82567d6694e83997d15dfc9d9f238', 'request_id': 'req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:56.062854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] The token cache is being invalidate because project f1f82567d6694e83997d15dfc9d9f238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:22:56.063151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:56.063721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be12a5b7-dbbb-476c-9d9d-4d44148c0c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:22:56.065153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 755/1510] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:22:55 2022] DELETE /identity/v3/projects/f1f82567d6694e83997d15dfc9d9f238 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:23:05.183228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=62 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:23:06.061017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:30.493482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e3a1d-6979-4b89-bec6-ddf233835674 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:30.494147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e3a1d-6979-4b89-bec6-ddf233835674 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:30.494147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e3a1d-6979-4b89-bec6-ddf233835674 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:30.495328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:30.500672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b911-6789-4827-a6a7-f00e32bc4469 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:30.501182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b911-6789-4827-a6a7-f00e32bc4469 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:30.501286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b911-6789-4827-a6a7-f00e32bc4469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:30.590886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-4eb3b911-6789-4827-a6a7-f00e32bc4469 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:30.592131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4eb3b911-6789-4827-a6a7-f00e32bc4469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:30.699425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:30.853572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e3a1d-6979-4b89-bec6-ddf233835674 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:30.856120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b43e3a1d-6979-4b89-bec6-ddf233835674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:30.982790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e3a1d-6979-4b89-bec6-ddf233835674 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:30.988449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84aa1fb-51ed-4575-90a8-60748e030f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:30.989071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84aa1fb-51ed-4575-90a8-60748e030f83 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:30.989357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84aa1fb-51ed-4575-90a8-60748e030f83 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:30.989643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84aa1fb-51ed-4575-90a8-60748e030f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:30.997863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84aa1fb-51ed-4575-90a8-60748e030f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:30.999489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84aa1fb-51ed-4575-90a8-60748e030f83 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:31.009569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 757/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:31.848722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f53b4-1194-4232-89a8-c95b48478b3c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.849389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f53b4-1194-4232-89a8-c95b48478b3c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.849389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f53b4-1194-4232-89a8-c95b48478b3c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:31.850606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:24:31.856009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dd565-8c59-4a3d-bb98-3cb4eb78dac8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:31.856540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dd565-8c59-4a3d-bb98-3cb4eb78dac8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:31.856651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dd565-8c59-4a3d-bb98-3cb4eb78dac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:31.878723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5a0dd565-8c59-4a3d-bb98-3cb4eb78dac8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:31.880008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5a0dd565-8c59-4a3d-bb98-3cb4eb78dac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:31.888757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:32.045314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f53b4-1194-4232-89a8-c95b48478b3c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:32.059867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f53b4-1194-4232-89a8-c95b48478b3c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:32.065270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc955c-300b-4382-bce0-29b1868f20b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:32.065882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc955c-300b-4382-bce0-29b1868f20b3 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:32.066162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc955c-300b-4382-bce0-29b1868f20b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:32.066434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc955c-300b-4382-bce0-29b1868f20b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:32.075185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc955c-300b-4382-bce0-29b1868f20b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:32.076820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc955c-300b-4382-bce0-29b1868f20b3 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:32.087549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 758/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:33.024507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c708b-095b-40b7-8389-5d6a347a878e None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:33.025055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c708b-095b-40b7-8389-5d6a347a878e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:33.025055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c708b-095b-40b7-8389-5d6a347a878e None None] PATH_INFO: `/v3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:33.026361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 759/1517] 10.222.0.16 () {54 vars in 925 bytes} [Mon Aug 29 21:24:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:24:33.032372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742ba7-49d9-4b8a-baf9-d8d68ca29b0a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:33.032848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742ba7-49d9-4b8a-baf9-d8d68ca29b0a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:33.032964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9742ba7-49d9-4b8a-baf9-d8d68ca29b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:33.098670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f9742ba7-49d9-4b8a-baf9-d8d68ca29b0a None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:33.099920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f9742ba7-49d9-4b8a-baf9-d8d68ca29b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:33.119420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:33.176765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 759/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 29 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:37.291573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d22e6-06af-4fb3-849f-cc5ae109242b None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.292134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d22e6-06af-4fb3-849f-cc5ae109242b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.292134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d22e6-06af-4fb3-849f-cc5ae109242b None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.293406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 760/1519] 10.222.0.65 () {54 vars in 961 bytes} [Mon Aug 29 21:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:37.298583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91a01-ca5c-48e3-a069-cc231dfeac71 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.299018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91a01-ca5c-48e3-a069-cc231dfeac71 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.299142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d91a01-ca5c-48e3-a069-cc231dfeac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.320268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-08d91a01-ca5c-48e3-a069-cc231dfeac71 None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:37.321491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-08d91a01-ca5c-48e3-a069-cc231dfeac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:37.330187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 760/1520] 10.222.0.65 () {58 vars in 1058 bytes} [Mon Aug 29 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:37.337959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d22e6-06af-4fb3-849f-cc5ae109242b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:37.351588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d22e6-06af-4fb3-849f-cc5ae109242b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:37.356985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00de1f1-83d0-44d0-9e43-407fe50e953b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:37.357622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00de1f1-83d0-44d0-9e43-407fe50e953b service placement] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:37.357911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00de1f1-83d0-44d0-9e43-407fe50e953b service placement] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:37.358206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00de1f1-83d0-44d0-9e43-407fe50e953b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:37.367718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00de1f1-83d0-44d0-9e43-407fe50e953b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:37.369383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00de1f1-83d0-44d0-9e43-407fe50e953b service placement] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:37.379456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 761/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:43.357072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f479d-c839-46a1-aad0-67444756fe73 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:43.357662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f479d-c839-46a1-aad0-67444756fe73 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:43.357662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f479d-c839-46a1-aad0-67444756fe73 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:43.359016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:43.364519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd991c1d-68eb-48bf-921f-37bf97ec7c06 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:43.364996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd991c1d-68eb-48bf-921f-37bf97ec7c06 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:43.365141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd991c1d-68eb-48bf-921f-37bf97ec7c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:43.387580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-fd991c1d-68eb-48bf-921f-37bf97ec7c06 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:43.388909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fd991c1d-68eb-48bf-921f-37bf97ec7c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:43.397475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:43.563101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f479d-c839-46a1-aad0-67444756fe73 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:43.578874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f479d-c839-46a1-aad0-67444756fe73 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:43.585465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aacd2ce-9b4f-45e3-a5aa-e3ad3239257e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:43.586074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacd2ce-9b4f-45e3-a5aa-e3ad3239257e service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:43.586353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacd2ce-9b4f-45e3-a5aa-e3ad3239257e service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:43.586627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacd2ce-9b4f-45e3-a5aa-e3ad3239257e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:43.595365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacd2ce-9b4f-45e3-a5aa-e3ad3239257e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:43.597274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aacd2ce-9b4f-45e3-a5aa-e3ad3239257e service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:43.606849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 762/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:44.452774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff93c18-94c1-4bac-9295-b6e399b0623b None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:44.453389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff93c18-94c1-4bac-9295-b6e399b0623b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:44.453389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff93c18-94c1-4bac-9295-b6e399b0623b None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:44.454581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 21:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:44.460261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d78557-c85d-494e-9fc8-15b1897f7794 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:44.460744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d78557-c85d-494e-9fc8-15b1897f7794 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:44.460855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d78557-c85d-494e-9fc8-15b1897f7794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:44.480485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-99d78557-c85d-494e-9fc8-15b1897f7794 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:44.481737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-99d78557-c85d-494e-9fc8-15b1897f7794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:44.490656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 21:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:44.648840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff93c18-94c1-4bac-9295-b6e399b0623b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:44.664246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff93c18-94c1-4bac-9295-b6e399b0623b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:44.670018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e2e29-5497-4a13-b40d-f32036160125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:44.670681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e2e29-5497-4a13-b40d-f32036160125 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:44.670946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e2e29-5497-4a13-b40d-f32036160125 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:44.671221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e2e29-5497-4a13-b40d-f32036160125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:44.681765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e2e29-5497-4a13-b40d-f32036160125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:44.683419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e2e29-5497-4a13-b40d-f32036160125 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:44.694834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 764/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:48.633748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95e02-5db5-4c88-a2cd-69f1af1ac40c None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:48.634335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95e02-5db5-4c88-a2cd-69f1af1ac40c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:48.634532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95e02-5db5-4c88-a2cd-69f1af1ac40c None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:48.636449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:48.642102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399023e-e724-4c92-85a4-32a1e71a6353 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:48.642680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399023e-e724-4c92-85a4-32a1e71a6353 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:48.642792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399023e-e724-4c92-85a4-32a1e71a6353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:48.664497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b399023e-e724-4c92-85a4-32a1e71a6353 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:48.665739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b399023e-e724-4c92-85a4-32a1e71a6353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:48.673848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:48.963464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95e02-5db5-4c88-a2cd-69f1af1ac40c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:48.965822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-67d95e02-5db5-4c88-a2cd-69f1af1ac40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:49.078042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95e02-5db5-4c88-a2cd-69f1af1ac40c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:49.081921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2b7f0-907c-4219-aaf6-fca30b0d86c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:49.082819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b7f0-907c-4219-aaf6-fca30b0d86c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:49.083101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b7f0-907c-4219-aaf6-fca30b0d86c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:49.083377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2b7f0-907c-4219-aaf6-fca30b0d86c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:49.093925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b7f0-907c-4219-aaf6-fca30b0d86c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:49.097050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2b7f0-907c-4219-aaf6-fca30b0d86c5 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:49.107192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 765/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:24:51.109439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb67ae5-b0a2-4291-b69e-daa80d8e9fb4 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:51.110180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb67ae5-b0a2-4291-b69e-daa80d8e9fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:51.110180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb67ae5-b0a2-4291-b69e-daa80d8e9fb4 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:51.111349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:51.117409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22089fd-5e7e-4826-be61-f843cab61304 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:51.117951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22089fd-5e7e-4826-be61-f843cab61304 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:51.117951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22089fd-5e7e-4826-be61-f843cab61304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:51.139508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c22089fd-5e7e-4826-be61-f843cab61304 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:51.140800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c22089fd-5e7e-4826-be61-f843cab61304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:51.149305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:51.436794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb67ae5-b0a2-4291-b69e-daa80d8e9fb4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:51.448568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb67ae5-b0a2-4291-b69e-daa80d8e9fb4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:51.452750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503823e-5f9a-4865-ae78-44bfb9236567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:51.453409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503823e-5f9a-4865-ae78-44bfb9236567 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:51.453693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503823e-5f9a-4865-ae78-44bfb9236567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:51.453978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503823e-5f9a-4865-ae78-44bfb9236567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:51.465060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503823e-5f9a-4865-ae78-44bfb9236567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:51.466592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503823e-5f9a-4865-ae78-44bfb9236567 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:51.477819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 767/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:54.806754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fc15-4faa-4492-a4af-0e66db364833 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:54.807343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fc15-4faa-4492-a4af-0e66db364833 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:54.807343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4fc15-4faa-4492-a4af-0e66db364833 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:54.808679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:54.814088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6028-a285-4a9c-8503-6da509f76cd9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:54.814534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6028-a285-4a9c-8503-6da509f76cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:54.814534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd6028-a285-4a9c-8503-6da509f76cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:54.835542 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6bcd6028-a285-4a9c-8503-6da509f76cd9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:54.836789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6bcd6028-a285-4a9c-8503-6da509f76cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:54.844645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:55.406517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fc15-4faa-4492-a4af-0e66db364833 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:55.408477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-24d4fc15-4faa-4492-a4af-0e66db364833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:55.476464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:55.492765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4fc15-4faa-4492-a4af-0e66db364833 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:55.496629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dd7ee-da29-4f6f-8c76-e304072260db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:55.497241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd7ee-da29-4f6f-8c76-e304072260db service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:55.497519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd7ee-da29-4f6f-8c76-e304072260db service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:55.497793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd7ee-da29-4f6f-8c76-e304072260db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:55.507177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dd7ee-da29-4f6f-8c76-e304072260db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:55.508644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dd7ee-da29-4f6f-8c76-e304072260db service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:55.518006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 768/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:24:57.476910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f927c-7348-4713-8f54-c7c8704ada69 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:57.477466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f927c-7348-4713-8f54-c7c8704ada69 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:57.477466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f927c-7348-4713-8f54-c7c8704ada69 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:57.478786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 21:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:24:57.484313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4798a6-3875-4410-a1aa-416a89fe2ab9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:57.484758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4798a6-3875-4410-a1aa-416a89fe2ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:57.484878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4798a6-3875-4410-a1aa-416a89fe2ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:57.504392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0e4798a6-3875-4410-a1aa-416a89fe2ab9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:24:57.505599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0e4798a6-3875-4410-a1aa-416a89fe2ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:57.513905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 21:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:24:58.077157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f927c-7348-4713-8f54-c7c8704ada69 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:24:58.084224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0d4f927c-7348-4713-8f54-c7c8704ada69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:24:58.086842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=84 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:24:58.092591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f927c-7348-4713-8f54-c7c8704ada69 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:24:58.096285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a14b2a-426d-4ab8-a816-beef6568b746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:24:58.096943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a14b2a-426d-4ab8-a816-beef6568b746 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:24:58.097874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a14b2a-426d-4ab8-a816-beef6568b746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:24:58.098231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a14b2a-426d-4ab8-a816-beef6568b746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:24:58.108328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a14b2a-426d-4ab8-a816-beef6568b746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:24:58.109783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a14b2a-426d-4ab8-a816-beef6568b746 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:24:58.127326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 770/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:25:07.499445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=11 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:25:08.130775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:25.046186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fd975-c11f-4478-b512-862403a456a8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.047290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fd975-c11f-4478-b512-862403a456a8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.047290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fd975-c11f-4478-b512-862403a456a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.071215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9f9fd975-c11f-4478-b512-862403a456a8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.072841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9f9fd975-c11f-4478-b512-862403a456a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.086765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 770/1540] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.097666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f4c7f-42b7-4521-b8a7-b0087d94a380 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.104799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f4c7f-42b7-4521-b8a7-b0087d94a380 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.104799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f4c7f-42b7-4521-b8a7-b0087d94a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.105021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.105021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.105021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.139242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.141064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-458f4c7f-42b7-4521-b8a7-b0087d94a380 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.142460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.142677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-458f4c7f-42b7-4521-b8a7-b0087d94a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.163344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 771/1541] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:25.167985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 771/1542] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.173673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.175451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.175750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.176121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.192479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0f590-259a-4546-afef-1b088fd443a4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.207940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.208989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.210058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.210683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.211596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:25.212180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.214093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eaf4e-cd2d-42d7-a36c-c59657febb17 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.226381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.227738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.250264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 772/1543] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:26:25.253494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.254121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.254736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.255705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 772/1544] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.260661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.283076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932498c4-3abd-4f2c-b22d-8731d7f44eea None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.296689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.298790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.298929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.299532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.299821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.300128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.300844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:25.301150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.302230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:26:25.302230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:26:25.302230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:26:25.303152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8380edc-a0ed-4c95-9370-5d3c859bd10f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.312933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 773/1545] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.314654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.328853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 773/1546] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:26:25.331098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.331401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.331705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.334285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8622f-c0d3-4fa1-b107-38075777c2ac None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.353009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.354432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.354960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.355633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.361211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.363732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:26:25.363732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:26:25.363732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:26:25.365482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367c85f-d40d-4b46-986f-4ac095638edf admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.375917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.379937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.402378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 774/1547] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.407608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.430427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dd49b-da0d-4520-859d-33adfd39097e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.446368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe368a31-5d39-4453-be16-31465c39bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.449039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368a31-5d39-4453-be16-31465c39bc5e admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.449945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368a31-5d39-4453-be16-31465c39bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.451029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe368a31-5d39-4453-be16-31465c39bc5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.452771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe368a31-5d39-4453-be16-31465c39bc5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.457964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe368a31-5d39-4453-be16-31465c39bc5e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.533708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 774/1548] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:25.535388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.535728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.536160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.567152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.569442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.592268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 775/1549] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/projects => generated 405 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:25.593964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739c1c0-80f5-4d18-a44a-2a15e908e372 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.594255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739c1c0-80f5-4d18-a44a-2a15e908e372 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.594543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739c1c0-80f5-4d18-a44a-2a15e908e372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.624988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0739c1c0-80f5-4d18-a44a-2a15e908e372 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.626627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0739c1c0-80f5-4d18-a44a-2a15e908e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.666323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 775/1550] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.667772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.683337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7a5ca-b75c-48bf-8124-4d59153f375a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.713136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 776/1551] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.714440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.714766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.715030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.742177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.743404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.752274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 777/1552] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.753155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.763220 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.763857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.764321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.764608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.765268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:25.765576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.767766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee26145-8e30-40c3-acc5-0aa1c61881a7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.773818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcf452-f8e9-49ad-8960-5dea47939789 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.780046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.781101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.781555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.782079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.788569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 776/1553] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:25.788782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.788889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d209e5-a9e9-4fee-9ea5-2fb14b000491 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.789001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:26:25.789001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:26:25.789001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:26:25.789242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d209e5-a9e9-4fee-9ea5-2fb14b000491 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.789242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d209e5-a9e9-4fee-9ea5-2fb14b000491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.789398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3880fe-4cc7-4f91-bc9d-2d4a4e609dbf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.815503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-10d209e5-a9e9-4fee-9ea5-2fb14b000491 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.819693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-10d209e5-a9e9-4fee-9ea5-2fb14b000491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.825065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 777/1554] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.826286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.826565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.826841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.848652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.849882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.858504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 778/1555] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.863223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.869686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690ea25-4f8c-4fdc-bc63-55d10022fc27 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.874782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8955639-d852-445b-ab3e-49b48c3252cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:25.875405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955639-d852-445b-ab3e-49b48c3252cd admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.875684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955639-d852-445b-ab3e-49b48c3252cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.875996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955639-d852-445b-ab3e-49b48c3252cd admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.877157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8955639-d852-445b-ab3e-49b48c3252cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:25.879283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8955639-d852-445b-ab3e-49b48c3252cd admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:25.894747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 778/1556] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:25.899396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8dd6f-d797-46ed-9369-195b9a5faf82 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.899396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8dd6f-d797-46ed-9369-195b9a5faf82 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.899396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8dd6f-d797-46ed-9369-195b9a5faf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.920234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-acd8dd6f-d797-46ed-9369-195b9a5faf82 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.921496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-acd8dd6f-d797-46ed-9369-195b9a5faf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.930666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 779/1557] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.932314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.932314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.932682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.945508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:25.954278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 779/1558] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:25.958215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad9a6-4565-4c57-8e92-bc20f039edfd None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:25.958215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad9a6-4565-4c57-8e92-bc20f039edfd None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:25.958215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad9a6-4565-4c57-8e92-bc20f039edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:25.965098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.966345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.978836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 780/1559] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:25.980375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:25.990541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-425ad9a6-4565-4c57-8e92-bc20f039edfd None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:25.995141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ce03e-101d-4b51-b2ff-18bb396895bf None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:25.995263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-425ad9a6-4565-4c57-8e92-bc20f039edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:25.999429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.000076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.000475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.000995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.001206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 780/1560] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:26.007160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.007340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c478954-871b-426c-939c-68ded1930dfe None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.007340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c478954-871b-426c-939c-68ded1930dfe None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.007340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c478954-871b-426c-939c-68ded1930dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.007608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.029692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9c478954-871b-426c-939c-68ded1930dfe None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:26.033293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9c478954-871b-426c-939c-68ded1930dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:26.043894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 781/1561] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:26.045857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.046268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.046983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.080962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:26.082130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:26.090677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 782/1562] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:26.091540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.102502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea69985-0649-41bc-ac71-0de92651fdb9 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.107478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.108103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.108381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.108652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.109300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.111450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.122738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 783/1563] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:26.124116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.137333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218336f7-3533-4a9f-9e75-670bf264149a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.140172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 781/1564] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:25 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:26.141464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.216771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:26.226832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.227595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.227879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.228243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.228864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:26.229172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.230816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.238816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5116438-46d3-4659-bf65-4584127272c9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.246693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.247776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.248282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.248744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.257823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 784/1565] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:26.257823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.258045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.258045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.271497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be26a84-6830-4dd4-83b8-0e1c8775115c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.277647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 782/1566] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:26.277647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.280460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.281329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.281329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.281699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.282364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:26.283318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.285204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.290129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63583a05-6c4a-4f0b-b6ac-29bd931765da admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.295520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.295821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.296139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.296418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.297070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:26.297378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.298439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 785/1567] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:26.299484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.299620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.317226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 783/1568] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:26.318173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.318341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f44a6-9996-4905-b664-188b800e9aa7 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.328756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.333685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.333685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.333685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.333685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:26.333685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.333685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.334994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b10434-1bfc-4a98-9aa5-4f027c267ad7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.341888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.342951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.343416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.344048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.344554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:26.344887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.347249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.351436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 786/1569] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:26.352363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.365155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e077db7-fe0a-4e17-aeee-4e580585375d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.375420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 784/1570] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:26.375420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.375731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.375731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.375731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.375731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.376227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.378324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.385301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ca996-303b-4880-bcce-7879dcdc3859 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.390634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.391384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.391668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.392103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.392899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.394874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.396410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 787/1571] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:26.398349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.414456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1cbbc-59fe-4031-b673-8c7e7f845baf admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.419727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.420420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.426376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.426376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.426376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.426376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.492171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 785/1572] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:26.493114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.511906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af715a01-fe89-4539-9da2-ab00076cff0b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.518310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.518842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.518842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.519202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.519795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.521710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.535729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 788/1573] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:26.540535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.540670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 786/1574] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:26.540670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.552047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014620-3f70-4286-8bbb-25df7dcb5317 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.637041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ea79c-de78-4fa7-a574-9a6d661c8de6 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.644044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.645052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.645443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.645907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.646654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.648531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.648920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.649556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.650063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.650551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.651329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.653353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.661033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:26.818914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 789/1575] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/projects => generated 365 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:26.819919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.853667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 787/1576] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/projects => generated 377 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:26.853667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:26.913488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb3d73-5d44-4e3f-b4db-6a5f174da3c6 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.933615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8dad0-5e6c-4a92-b263-6ccedab70ef7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:26.938554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.939290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.939626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.939963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.940635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.942578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:26.979259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:26.982835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:26.982835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:26.982835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:26.982835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:26.983267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.059439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:27.071047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 788/1577] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/projects => generated 391 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.072452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.141442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 790/1578] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:26 2022] POST /identity/v3/projects => generated 381 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.141442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.167910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b062a-aed4-4d2f-b540-71efb696a975 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.214261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952403af-66b2-4129-b731-b9278e5e7ece admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.219878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.220653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.226638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.226638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.226638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.226638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.231757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.232491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.232975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.233145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] PATH_INFO: `/v3/projects/c852db97ba734c7088df6af689b41d16/users/e135d9b5894147e6a391d75d8896f094/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.238023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 791/1579] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:27.238023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.254120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8826e-29a8-4ca4-8d5f-0860c045a8be admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.254779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c852db97ba734c7088df6af689b41d16, user_id=e135d9b5894147e6a391d75d8896f094, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.258768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.260540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.261315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.261594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.262055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] PATH_INFO: `/v3/projects/d9b884bbb6b647dfa7ed3a3b212fd44d/users/9140be8e2a7a4361b1afdbe32738b60e/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.283296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b884bbb6b647dfa7ed3a3b212fd44d, user_id=9140be8e2a7a4361b1afdbe32738b60e, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.284452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:26:27.284452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:26:27.284452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:26:27.286696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.289283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:27.532567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 789/1580] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:27 2022] PUT /identity/v3/projects/c852db97ba734c7088df6af689b41d16/users/e135d9b5894147e6a391d75d8896f094/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:27.533516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.543596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 792/1581] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:27 2022] PUT /identity/v3/projects/d9b884bbb6b647dfa7ed3a3b212fd44d/users/9140be8e2a7a4361b1afdbe32738b60e/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:27.544857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.602107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a98121-5d3b-4c27-9762-9802867395d5 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.606931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.607547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.610565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.610565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.610565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.611505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.611624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eff572-b550-4d9f-8051-d13d3e47c4a3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.616408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.617025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.617307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.617584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.618216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.620944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.692773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 790/1582] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.693606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.703354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70420c3-cf46-41cd-862e-ad9d0ee0df3d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.708187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.713869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.713869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.713869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.713869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.713869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.747183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 793/1583] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.747183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.755679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef6478-ed87-4e51-abc2-aec9bc6dffb8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.760840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.761542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.761918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.762282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.763003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.765417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.795583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:27.801479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 791/1584] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.802530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.813603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f78ea8-2b23-4ad1-a486-228258c4ee34 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.820583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461cd65-e1c2-4c61-bc4d-3911aa92f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.822891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461cd65-e1c2-4c61-bc4d-3911aa92f27f admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.822891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461cd65-e1c2-4c61-bc4d-3911aa92f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.822891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461cd65-e1c2-4c61-bc4d-3911aa92f27f admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.822891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461cd65-e1c2-4c61-bc4d-3911aa92f27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.824408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461cd65-e1c2-4c61-bc4d-3911aa92f27f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.870075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 794/1585] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.871022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.883135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5af6b-07fc-4aea-a9bd-3721e4b8f83e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.887760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8309d-a2ac-4f1b-bf50-8c744e84425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.888400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8309d-a2ac-4f1b-bf50-8c744e84425e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.888678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8309d-a2ac-4f1b-bf50-8c744e84425e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.888951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8309d-a2ac-4f1b-bf50-8c744e84425e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.889588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8309d-a2ac-4f1b-bf50-8c744e84425e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.891228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8309d-a2ac-4f1b-bf50-8c744e84425e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.902101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 795/1586] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:27.903317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.903600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.903907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.925926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 792/1587] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:27.928074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.928074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.928074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.930220 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:27.932169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:27.943773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 796/1588] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:27.944689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.956988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63afff8-fa07-4b88-8678-26ada5d94428 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:27.965035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:27.965678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:27.965996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:27.966241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:27.966894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:27.968633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:27.972143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] MFA Rules not processed for user `9140be8e2a7a4361b1afdbe32738b60e`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:27.973193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:27.989427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 797/1589] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:27.989427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:27.997382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725a1bc-0f5a-46a0-86b7-9a995673a90a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.002387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.003029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.003306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.003586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.004435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.006382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.021658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 798/1590] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.022816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.034346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d549f6b-9ef0-4f24-8a3f-c04dfc2fd5c8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.039450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.040269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.040598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.040833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.041472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.043091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.063050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 799/1591] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.063050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.063351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 793/1592] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:28.063351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.075583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb542a9c-3816-47d9-8581-6a6d95e25ddd None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.077007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800180f0-cb2c-4921-9a2e-1c3ce295e623 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.080665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.081355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.081634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.086636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] PATH_INFO: `/v3/projects/8f58a8e730ad4824b8ea4d5a18908675/users/090d9d685305459fb3a72e75f1e597e9/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.086784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.086784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.086784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.087237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.088391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.091531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.100159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f58a8e730ad4824b8ea4d5a18908675, user_id=090d9d685305459fb3a72e75f1e597e9, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.102956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.111823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 800/1593] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.113556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.130844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e127-69de-4d34-ac70-b7780c8e1959 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.138553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.139590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.140122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.140621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.141759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.144866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.159907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 801/1594] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.160865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.162756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:28.192889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 794/1595] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:28 2022] PUT /identity/v3/projects/8f58a8e730ad4824b8ea4d5a18908675/users/090d9d685305459fb3a72e75f1e597e9/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:28.194462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.245428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ad089-7560-4fcf-ae08-756982ac976f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.251381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:28.283547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99249c-90ac-4b88-9a2e-30e6516f2269 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.290003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.290633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.291120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.291400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.292082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.293870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.307054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.307711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.308024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.308151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 795/1596] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.308337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.309029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.324109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971d74b-eefa-45ba-b422-0db8a350a448 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.327104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.328639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.330455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.331452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.331735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.332043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.332690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.334310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.347337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 802/1597] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:28.348312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.349321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 796/1598] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.350627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.358277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752dae-fe08-4408-9ecf-088470cf2775 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.363612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1cb72-bed3-46c5-8c5d-163d5c1a94cc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.364065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.364701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.364981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.365256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.365904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.367651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.369212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.369841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.370136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.370411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.371217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.373326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.387662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 803/1599] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.387662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.401806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38a4ff-7a80-44dc-805f-f6fcc2483cfc admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.406974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.407649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.407955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.408429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.409077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.410841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.421529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 804/1600] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.423012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.438733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d968deb-361f-4139-8493-f0898bee7532 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.444493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26423d-4125-4b54-a646-622551405401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.445110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26423d-4125-4b54-a646-622551405401 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.445387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26423d-4125-4b54-a646-622551405401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.456909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26423d-4125-4b54-a646-622551405401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.456909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26423d-4125-4b54-a646-622551405401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.456909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26423d-4125-4b54-a646-622551405401 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.461317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 805/1601] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:28.462873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.463159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.463430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.497794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] MFA Rules not processed for user `090d9d685305459fb3a72e75f1e597e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:28.507502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:28.567536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 806/1602] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:28.568445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.582945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140256dc-b0be-4050-af9d-3ddd55b6cd11 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.588166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.588859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.589141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.589417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] PATH_INFO: `/v3/projects/8c7b1ec1f3fa4f38ac616f3d03ec5a10/users/fe195a70eb1a470e8f6c7eb1da7dc67c/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.607418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7b1ec1f3fa4f38ac616f3d03ec5a10, user_id=fe195a70eb1a470e8f6c7eb1da7dc67c, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.610253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.627535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 797/1603] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:28 2022] POST /identity/v3/projects => generated 403 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:28.628609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.641869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:28.695064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b604-5f6c-4607-873a-27cd69513b0b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.699924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.700593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.700873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.701175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] PATH_INFO: `/v3/projects/04b4e36569b9430bbfab0c39b6041fdf/users/ab1d27ec74814218a1f214b120333767/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.716577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b4e36569b9430bbfab0c39b6041fdf, user_id=ab1d27ec74814218a1f214b120333767, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.719574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.743719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 807/1604] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:28 2022] PUT /identity/v3/projects/8c7b1ec1f3fa4f38ac616f3d03ec5a10/users/fe195a70eb1a470e8f6c7eb1da7dc67c/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:28.748326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.794607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:28.812691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978855d7-2ffe-4a65-8284-3abff916d481 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.819844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.820572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.820855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.821204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] PATH_INFO: `/v3/projects/d8d35a885c494e11b42f556554f3698b/users/d3c5edb153624912b87d61dd7ad5e4f3/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.838423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 798/1605] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:28 2022] PUT /identity/v3/projects/04b4e36569b9430bbfab0c39b6041fdf/users/ab1d27ec74814218a1f214b120333767/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:28.839358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:28.842895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d35a885c494e11b42f556554f3698b, user_id=d3c5edb153624912b87d61dd7ad5e4f3, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.845669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.917164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa8810-92d9-4e52-bd2b-2b5389596495 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:28.922152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:28.922828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:28.923140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:28.924027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] PATH_INFO: `/v3/projects/bd7bb76694c2499895f24e611c644a27/users/b7e0a78238e94fa4a8209550c901a9a5/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:28.939258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7bb76694c2499895f24e611c644a27, user_id=b7e0a78238e94fa4a8209550c901a9a5, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:28.941981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:28.987721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 808/1606] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:28 2022] PUT /identity/v3/projects/d8d35a885c494e11b42f556554f3698b/users/d3c5edb153624912b87d61dd7ad5e4f3/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:28.987721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.046623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 799/1607] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:28 2022] PUT /identity/v3/projects/bd7bb76694c2499895f24e611c644a27/users/b7e0a78238e94fa4a8209550c901a9a5/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:29.047616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.064168 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22c26-e642-4aa7-84a2-7554df19fb3b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.120361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64cb5c-a62a-4ff6-9704-5f2379116fda admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.125423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.126420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.126420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.126654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.127229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.128880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bde826-4d5f-40e6-b0d4-2f2244c98812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.129575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde826-4d5f-40e6-b0d4-2f2244c98812 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.129919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde826-4d5f-40e6-b0d4-2f2244c98812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.130280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde826-4d5f-40e6-b0d4-2f2244c98812 admin admin] PATH_INFO: `/v3/projects/8695633e1e2744b6b6554536f1109fc6/users/68c15654f5a14850a1763a020c17c849/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.132121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.148430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde826-4d5f-40e6-b0d4-2f2244c98812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8695633e1e2744b6b6554536f1109fc6, user_id=68c15654f5a14850a1763a020c17c849, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.154401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde826-4d5f-40e6-b0d4-2f2244c98812 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.254656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 800/1608] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:29.255579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.260741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:29.288700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 809/1609] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:28 2022] PUT /identity/v3/projects/8695633e1e2744b6b6554536f1109fc6/users/68c15654f5a14850a1763a020c17c849/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:29.294180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c161-a460-4b6f-8a87-d56e4a1978d8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.294676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c161-a460-4b6f-8a87-d56e4a1978d8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.295137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c161-a460-4b6f-8a87-d56e4a1978d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.328808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2bd5-c0fd-4430-8634-5fff2238d7c6 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.333776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f833c161-a460-4b6f-8a87-d56e4a1978d8 None None] MFA Rules not processed for user `fe195a70eb1a470e8f6c7eb1da7dc67c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.335704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f833c161-a460-4b6f-8a87-d56e4a1978d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:29.390060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:29.404942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edda4f-8b2d-4123-bd8a-c5fd889c233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.405577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edda4f-8b2d-4123-bd8a-c5fd889c233a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.405914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edda4f-8b2d-4123-bd8a-c5fd889c233a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.406199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edda4f-8b2d-4123-bd8a-c5fd889c233a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.406990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edda4f-8b2d-4123-bd8a-c5fd889c233a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.409110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edda4f-8b2d-4123-bd8a-c5fd889c233a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.414647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 810/1610] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:29.416028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c2196-e1fe-41c0-b535-8e0fdeb70076 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.420618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c2196-e1fe-41c0-b535-8e0fdeb70076 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.420618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c2196-e1fe-41c0-b535-8e0fdeb70076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.462972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-eb3c2196-e1fe-41c0-b535-8e0fdeb70076 None None] MFA Rules not processed for user `ab1d27ec74814218a1f214b120333767`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.465327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-eb3c2196-e1fe-41c0-b535-8e0fdeb70076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:29.531720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 801/1611] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/users => generated 308 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:29.533103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.533395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.533671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.557877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 811/1612] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:29.557877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976495-5cfa-4188-845f-e3ae7485e9bf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.557877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976495-5cfa-4188-845f-e3ae7485e9bf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.557877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976495-5cfa-4188-845f-e3ae7485e9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.558333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.558333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:29.566704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 802/1613] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:29.567638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.574733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1d976495-5cfa-4188-845f-e3ae7485e9bf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.576039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1d976495-5cfa-4188-845f-e3ae7485e9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:29.577475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24555d51-77cc-4a5c-8e22-b7de57f64847 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.582758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.583369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.583647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.583945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.585060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.585192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 812/1614] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:29.586753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.589144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0c1d-11c3-45da-998d-7fd47bf928df None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.589434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0c1d-11c3-45da-998d-7fd47bf928df None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.589719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a0c1d-11c3-45da-998d-7fd47bf928df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.608557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-7a0a0c1d-11c3-45da-998d-7fd47bf928df None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.609845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7a0a0c1d-11c3-45da-998d-7fd47bf928df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:29.617664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 813/1615] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:29.619239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2af8-3973-4ffa-a543-ef53a471b8eb None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.623409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2af8-3973-4ffa-a543-ef53a471b8eb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.623409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2af8-3973-4ffa-a543-ef53a471b8eb None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.623409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 814/1616] 10.222.0.65 () {54 vars in 962 bytes} [Mon Aug 29 21:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:29.623409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2af8-3973-4ffa-a543-ef53a471b8eb None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.635183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2af8-3973-4ffa-a543-ef53a471b8eb None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.639690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.640653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.641079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.643098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.643756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.645454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.655583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 815/1617] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:29.656415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.668223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c98d0-10ce-418e-8507-369067c09c49 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.673812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.674433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.674724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.674997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.675646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.677684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.783660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 803/1618] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:29.784674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.840524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b21f2-ac10-4153-a300-c8a036dfd80f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.845868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.846500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.846785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.847256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.857608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.859097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.869268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 804/1619] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:29.870882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.878464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ce954-3a7e-49ae-a1d9-59a5a5237509 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:29.882938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbde39f-5a64-413e-9a6e-fdd2e1e97827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:29.883545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde39f-5a64-413e-9a6e-fdd2e1e97827 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.883827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde39f-5a64-413e-9a6e-fdd2e1e97827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.884132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbde39f-5a64-413e-9a6e-fdd2e1e97827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.894338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde39f-5a64-413e-9a6e-fdd2e1e97827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:29.895731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbde39f-5a64-413e-9a6e-fdd2e1e97827 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:29.905457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 805/1620] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:29.906672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:29.906951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:29.907231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:29.946622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 816/1621] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/projects => generated 361 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:29.947797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:29.950723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:29.952007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:30.010180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3c31a-c54b-4933-8b14-4868beecfcab admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.015359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.016027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.016316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.016588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.030681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 806/1622] 10.222.0.65 () {58 vars in 1059 bytes} [Mon Aug 29 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:30.037616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.109004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a04fa-2f7c-4531-a723-b5760923c92e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.114201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.115694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.115694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.115694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] PATH_INFO: `/v3/projects/dfa546cb9de94e54bb1cc65fb41f3d33/users/643db4ab4e134d76b19e5121b472fc8d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.132754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa546cb9de94e54bb1cc65fb41f3d33, user_id=643db4ab4e134d76b19e5121b472fc8d, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.135683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.140399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.142992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.155106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 817/1623] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:30.155916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.165913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43d19e-d563-4d5a-8a20-e0f94b40fab5 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.171929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.173094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.173094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.173494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.174261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.175769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:30.176434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.189887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 818/1624] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:30.190705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.209045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 807/1625] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:30 2022] PUT /identity/v3/projects/dfa546cb9de94e54bb1cc65fb41f3d33/users/643db4ab4e134d76b19e5121b472fc8d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:30.209045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.227976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e167c-289f-40b5-ab6b-3e444e56a79e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.257735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:30.301030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.301960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.303117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.303117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.303580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.306247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.308717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a7be9-4f12-493a-97f3-fec18b456e27 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.315815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.316486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.316771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.317049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.414921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.414921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.414921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 808/1626] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:30.427708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.466971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9780a33-3413-4ae3-be69-020592b6b2d2 service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.479936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.480904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.481352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.481816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] PATH_INFO: `/v3/projects/bba0123e9f4c474793c1532073a08fb0/users/118bc888cd5d43c29f2180827502dcc6/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.500202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 819/1627] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:30 2022] POST /identity/v3/users => generated 287 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:30.501164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.526683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba0123e9f4c474793c1532073a08fb0, user_id=118bc888cd5d43c29f2180827502dcc6, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.529877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.530545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77526918-1d50-4277-acc5-da76f5248efc admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.537013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.537649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.537958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.538208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.538865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.540549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.568874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 820/1628] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:30.570348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.593602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34687ea-74f8-4423-82d9-862163b49030 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.602817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.604455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.604834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.605219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.606766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.609704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.615442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 809/1629] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:30 2022] PUT /identity/v3/projects/bba0123e9f4c474793c1532073a08fb0/users/118bc888cd5d43c29f2180827502dcc6/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:30.617745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.637127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 821/1630] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:30.639125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.656282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147d059-5415-4782-9b38-e352611dde5a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.721624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d06ad6-c2d2-4a82-9269-658689cbf617 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.727345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.728005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.728292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.728402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.728577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.729081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.729248 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.729408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.729650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] PATH_INFO: `/v3/projects/dfa546cb9de94e54bb1cc65fb41f3d33/users/643db4ab4e134d76b19e5121b472fc8d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.730924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.732143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa546cb9de94e54bb1cc65fb41f3d33, user_id=643db4ab4e134d76b19e5121b472fc8d, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.736386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.742502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 822/1631] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:30.743353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.750993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 810/1632] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:30 2022] PUT /identity/v3/projects/dfa546cb9de94e54bb1cc65fb41f3d33/users/643db4ab4e134d76b19e5121b472fc8d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:30.753036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:30.781147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0376-c7b1-4507-9608-b1a882e3592b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.874452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16db410-8bc6-4676-9c02-b34ad56f6700 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:30.884120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef207014-00e3-4852-87c7-ac6a541a6534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.885196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef207014-00e3-4852-87c7-ac6a541a6534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.885483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef207014-00e3-4852-87c7-ac6a541a6534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.885759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef207014-00e3-4852-87c7-ac6a541a6534 admin admin] PATH_INFO: `/v3/projects/bba0123e9f4c474793c1532073a08fb0/users/118bc888cd5d43c29f2180827502dcc6/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.888019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef207014-00e3-4852-87c7-ac6a541a6534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba0123e9f4c474793c1532073a08fb0, user_id=118bc888cd5d43c29f2180827502dcc6, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.889914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915e97c-467f-47c6-8700-ca7d43b5ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:30.890729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e97c-467f-47c6-8700-ca7d43b5ecec admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.890991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef207014-00e3-4852-87c7-ac6a541a6534 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.891138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e97c-467f-47c6-8700-ca7d43b5ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.891538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915e97c-467f-47c6-8700-ca7d43b5ecec admin admin] PATH_INFO: `/v3/projects/a84aa5e3024c4adb8b2724daf318900a/users/45069d56482443b4ac13fd54d4cfbf1b/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.904631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 811/1633] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:30 2022] PUT /identity/v3/projects/bba0123e9f4c474793c1532073a08fb0/users/118bc888cd5d43c29f2180827502dcc6/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:30.911005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471de42-82d6-4319-9661-c4b9fda51ecb None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:30.911005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471de42-82d6-4319-9661-c4b9fda51ecb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:30.911005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471de42-82d6-4319-9661-c4b9fda51ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:30.925176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e97c-467f-47c6-8700-ca7d43b5ecec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84aa5e3024c4adb8b2724daf318900a, user_id=45069d56482443b4ac13fd54d4cfbf1b, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:30.928316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915e97c-467f-47c6-8700-ca7d43b5ecec admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:30.944398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2471de42-82d6-4319-9661-c4b9fda51ecb None None] MFA Rules not processed for user `643db4ab4e134d76b19e5121b472fc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:31.272534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2471de42-82d6-4319-9661-c4b9fda51ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:31.272534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:31.274042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 823/1634] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:30 2022] PUT /identity/v3/projects/a84aa5e3024c4adb8b2724daf318900a/users/45069d56482443b4ac13fd54d4cfbf1b/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:31.274042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a518-7825-4855-ac48-07a0e7bb2172 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.274042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a518-7825-4855-ac48-07a0e7bb2172 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.274042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a518-7825-4855-ac48-07a0e7bb2172 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.275941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 812/1635] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:31.277243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 824/1636] 10.222.0.65 () {54 vars in 912 bytes} [Mon Aug 29 21:26:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:31.277243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10a518-7825-4855-ac48-07a0e7bb2172 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:31.278486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.278486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.278486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.278486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] MFA Rules not processed for user `118bc888cd5d43c29f2180827502dcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:31.278486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:31.279663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:31.279663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10a518-7825-4855-ac48-07a0e7bb2172 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:31.279663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0a91f-d00c-4403-ad77-19146424d68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:31.279663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0a91f-d00c-4403-ad77-19146424d68d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.280913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0a91f-d00c-4403-ad77-19146424d68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.280913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0a91f-d00c-4403-ad77-19146424d68d admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.280913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0a91f-d00c-4403-ad77-19146424d68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:31.280913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0a91f-d00c-4403-ad77-19146424d68d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:31.280913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 825/1637] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:31.280913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.286547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.286547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.292500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 813/1638] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:31.292500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:31.294594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:31.294594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:31.360188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea59d09-5fa3-4d96-a1d0-f155496b835e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:31.368864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf3e0c-4e38-4099-b559-edc12fc7cdb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:31.369956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf3e0c-4e38-4099-b559-edc12fc7cdb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.370564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf3e0c-4e38-4099-b559-edc12fc7cdb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.371150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf3e0c-4e38-4099-b559-edc12fc7cdb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.378374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 826/1639] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:31.384896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:31.397895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc7b0a-ed8f-4653-b0d4-3fe30cebd904 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:31.404263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:31.404956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.406604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.406604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] PATH_INFO: `/v3/projects/a84aa5e3024c4adb8b2724daf318900a/users/45069d56482443b4ac13fd54d4cfbf1b/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.408171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84aa5e3024c4adb8b2724daf318900a, user_id=45069d56482443b4ac13fd54d4cfbf1b, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:31.410964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:31.431641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 827/1640] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:31 2022] PUT /identity/v3/projects/a84aa5e3024c4adb8b2724daf318900a/users/45069d56482443b4ac13fd54d4cfbf1b/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:31.431641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:31.447101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf3e0c-4e38-4099-b559-edc12fc7cdb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:31.448610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf3e0c-4e38-4099-b559-edc12fc7cdb7 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:31.461295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 814/1641] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:31 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:31.462896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.463155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.463461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.527484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] MFA Rules not processed for user `45069d56482443b4ac13fd54d4cfbf1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:31.527484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:31.529805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf77ee-893f-45da-9b3e-f16bd848b648 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:31.538170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:31.540009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.541606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.541949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.671165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:31.674032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:31.680546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 815/1642] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:31.682152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:31.688059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 828/1643] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:31.719717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f435-60e4-434f-8528-d25ca1201a11 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:31.729112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:31.730549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:31.732308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:31.733296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:31.870142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:31.875914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:31.899588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 816/1644] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:32.055289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:32.071742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ec8f-13ba-4f3e-a0eb-6ec25dc268f4 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:32.082803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:32.085385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:32.085385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:32.085558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:32.107184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:32.109692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:32.123924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 829/1645] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:32.308770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:32.869964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:32.885116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae878c-62e7-4f2f-bb33-5ea2fe995bd5 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:32.889864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:32.890755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:32.891236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:32.891674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:32.953185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:33.013070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:33.014669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:33.025954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 817/1646] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:33.515648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:33.525036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:33.535184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf686f3-02b3-4596-ab02-a166ee0aff58 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:33.539685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e83c4-df3e-4668-9a88-2c3442498ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:33.540650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e83c4-df3e-4668-9a88-2c3442498ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:33.541096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e83c4-df3e-4668-9a88-2c3442498ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:33.541627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e83c4-df3e-4668-9a88-2c3442498ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:33.626165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e83c4-df3e-4668-9a88-2c3442498ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:33.629959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e83c4-df3e-4668-9a88-2c3442498ab7 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:33.662466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982240ca-ddf2-407c-9555-d33d2538b9d5 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:33.666702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 818/1647] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:26:33.677048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:33.677619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:33.677902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:33.678176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] PATH_INFO: `/v3/users/9140be8e2a7a4361b1afdbe32738b60e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:33.679389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9140be8e2a7a4361b1afdbe32738b60e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:33.680478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: WARNING py.warnings [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:26:33.680478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: warnings.warn(msg) Aug 29 21:26:33.680478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: Aug 29 21:26:33.681719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:33.976684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9140be8e2a7a4361b1afdbe32738b60e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:34.069098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9140be8e2a7a4361b1afdbe32738b60e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:34.090075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9140be8e2a7a4361b1afdbe32738b60e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:34.099195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:34.135760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9140be8e2a7a4361b1afdbe32738b60e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:34.143058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9140be8e2a7a4361b1afdbe32738b60e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:34.154310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 830/1648] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:33 2022] DELETE /identity/v3/users/9140be8e2a7a4361b1afdbe32738b60e => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:34.159098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804eb2-eea2-4256-b111-c90269d088ca None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.159429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804eb2-eea2-4256-b111-c90269d088ca None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.159843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79804eb2-eea2-4256-b111-c90269d088ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.217385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-79804eb2-eea2-4256-b111-c90269d088ca None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:34.219443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-79804eb2-eea2-4256-b111-c90269d088ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:34.326356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 819/1649] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:34.339098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:34.340947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:34.385591 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:34.422895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b595055-99d2-4ff8-8aa0-5a507b462274 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.423227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b595055-99d2-4ff8-8aa0-5a507b462274 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.423559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b595055-99d2-4ff8-8aa0-5a507b462274 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.424935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 820/1650] 10.222.0.65 () {54 vars in 962 bytes} [Mon Aug 29 21:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:26:34.451704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e91687-c293-4344-95d8-da1daa95ce37 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.452081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e91687-c293-4344-95d8-da1daa95ce37 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.452311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e91687-c293-4344-95d8-da1daa95ce37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.476140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8497-5ced-4410-87c3-718846cf72de admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:34.485393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeeec6b-6a21-46a0-9beb-c68237dd1863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:34.486172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeec6b-6a21-46a0-9beb-c68237dd1863 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.486349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeec6b-6a21-46a0-9beb-c68237dd1863 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.486620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeeec6b-6a21-46a0-9beb-c68237dd1863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.512979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d6e91687-c293-4344-95d8-da1daa95ce37 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:34.514264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d6e91687-c293-4344-95d8-da1daa95ce37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:34.519751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeeec6b-6a21-46a0-9beb-c68237dd1863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:34.521130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeeec6b-6a21-46a0-9beb-c68237dd1863 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:34.534636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 831/1651] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:34.647340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:34.681817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 821/1652] 10.222.0.65 () {58 vars in 1059 bytes} [Mon Aug 29 21:26:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:34.693015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36afda-a2a2-4fae-bbb7-d6cf5480e150 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.693467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36afda-a2a2-4fae-bbb7-d6cf5480e150 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.693812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36afda-a2a2-4fae-bbb7-d6cf5480e150 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.695584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 832/1653] 10.222.0.65 () {54 vars in 912 bytes} [Mon Aug 29 21:26:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:34.701518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.701892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.702265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.742249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:34.745640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:34.782739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36afda-a2a2-4fae-bbb7-d6cf5480e150 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:34.784700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2f36afda-a2a2-4fae-bbb7-d6cf5480e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:34.788906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 822/1654] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:26:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:34.929952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36afda-a2a2-4fae-bbb7-d6cf5480e150 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:34.935049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:34.936776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:34.936776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:34.936776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:34.962190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:34.964087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:35.004136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 833/1655] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:35.714841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:35.768300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:35.770962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:35.790718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6d2ab-678d-4fba-bcc5-ea3079dcbcd7 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:35.802000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:35.802000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:35.802000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:35.802000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] PATH_INFO: `/v3/projects/d9b884bbb6b647dfa7ed3a3b212fd44d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:35.802284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b884bbb6b647dfa7ed3a3b212fd44d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:35.802284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: WARNING py.warnings [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 21:26:35.802284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: warnings.warn(msg) Aug 29 21:26:35.802284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: Aug 29 21:26:35.804856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:36.035264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b884bbb6b647dfa7ed3a3b212fd44d', 'request_id': 'req-6fb3c472-82da-4204-8dc9-65693eab3089'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.057008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b884bbb6b647dfa7ed3a3b212fd44d', 'request_id': 'req-6fb3c472-82da-4204-8dc9-65693eab3089'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.223577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b884bbb6b647dfa7ed3a3b212fd44d', 'request_id': 'req-6fb3c472-82da-4204-8dc9-65693eab3089'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.240560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] The token cache is being invalidate because project d9b884bbb6b647dfa7ed3a3b212fd44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:36.240934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.241845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.244265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 823/1656] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:35 2022] DELETE /identity/v3/projects/d9b884bbb6b647dfa7ed3a3b212fd44d => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:36.247847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:36.261818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a1f71-7c50-42e0-b93f-0cd146738d39 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:36.415875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:36.416633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:36.416915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:36.417184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] PATH_INFO: `/v3/users/643db4ab4e134d76b19e5121b472fc8d` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:36.420754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=643db4ab4e134d76b19e5121b472fc8d)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:36.428684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:36.764757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643db4ab4e134d76b19e5121b472fc8d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.926245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643db4ab4e134d76b19e5121b472fc8d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.937382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643db4ab4e134d76b19e5121b472fc8d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.960864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643db4ab4e134d76b19e5121b472fc8d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.961961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ac686eef-982c-4bd2-9316-597aa11d0c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643db4ab4e134d76b19e5121b472fc8d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:36.970684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 834/1657] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:36 2022] DELETE /identity/v3/users/643db4ab4e134d76b19e5121b472fc8d => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:37.345969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:38.292623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:38.294079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:38.317732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3c472-82da-4204-8dc9-65693eab3089 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:38.492535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:38.494202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:38.494680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:38.495097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] PATH_INFO: `/v3/projects/dfa546cb9de94e54bb1cc65fb41f3d33` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:38.497019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa546cb9de94e54bb1cc65fb41f3d33)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:38.499343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:38.723518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa546cb9de94e54bb1cc65fb41f3d33', 'request_id': 'req-ae972d23-6999-47c8-b99f-da2dd687ce5d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:38.749630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa546cb9de94e54bb1cc65fb41f3d33', 'request_id': 'req-ae972d23-6999-47c8-b99f-da2dd687ce5d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:38.875528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa546cb9de94e54bb1cc65fb41f3d33', 'request_id': 'req-ae972d23-6999-47c8-b99f-da2dd687ce5d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:38.910460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] The token cache is being invalidate because project dfa546cb9de94e54bb1cc65fb41f3d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:38.911307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:38.913182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:38.922460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 824/1658] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:38 2022] DELETE /identity/v3/projects/dfa546cb9de94e54bb1cc65fb41f3d33 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:38.935005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc28e94-0f85-40fd-8be4-823c0018d611 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:38.935970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc28e94-0f85-40fd-8be4-823c0018d611 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:38.936462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc28e94-0f85-40fd-8be4-823c0018d611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:38.959566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:38.959566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:38.992699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:38.996175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1cc28e94-0f85-40fd-8be4-823c0018d611 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:38.998128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1cc28e94-0f85-40fd-8be4-823c0018d611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:39.120630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae972d23-6999-47c8-b99f-da2dd687ce5d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:39.150632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:39.152840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:39.157252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:39.157955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] PATH_INFO: `/v3/users/ab1d27ec74814218a1f214b120333767` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:39.166127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1d27ec74814218a1f214b120333767)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:39.168429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:39.174551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 835/1659] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:39.183444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:39.184230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:39.184521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:39.237175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:39.239272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:39.249313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 836/1660] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:39.254014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:39.265418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88db0c-e699-469c-b904-606f16141bcf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:39.393437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:39.393437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:39.393437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:39.393437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:39.396171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:39.396171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:39.396171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:39.396171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 837/1661] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:39.396171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:39.396171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77080-1737-43c6-82b2-bd2a60f688e7 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:39.396935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:39.396935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:39.396935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:39.396935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:39.397298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:39.397298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:39.613478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 838/1662] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:39 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:26:39.617418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:39.759845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1d27ec74814218a1f214b120333767'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:39.763779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1d27ec74814218a1f214b120333767'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:39.796377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dea2e-0ee8-4f9e-beba-34915e85ffa0 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:39.802531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:39.803176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:39.803476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:39.803739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:39.804421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:39.806398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:39.809097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1d27ec74814218a1f214b120333767'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:39.810099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1d27ec74814218a1f214b120333767'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:39.837643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-00e91a2e-600e-458d-8b6a-0d81a40f906c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1d27ec74814218a1f214b120333767'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:39.984079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 825/1663] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:38 2022] DELETE /identity/v3/users/ab1d27ec74814218a1f214b120333767 => generated 0 bytes in 1028 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:26:39.986970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:39.986970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:39.986970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.020125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:40.027228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 839/1664] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:39 2022] POST /identity/v3/users => generated 291 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:26:40.032034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:40.037729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:40.063293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057e3c7-adfe-4d51-aebb-09963e60f419 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:40.065368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:40.071786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:40.200003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 826/1665] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:40.234119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:40.238197 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:40.238920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.239570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:40.239702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.239702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.240127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:40.243387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:40.273853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 840/1666] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:40.277797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:40.303515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be7de5-60a7-4f48-861d-89be7a68e07e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:40.320260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:40.321344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.321773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.322223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.323009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:40.325227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:40.338405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05ed9d-4e77-47b2-83a9-d1ecf8fb0c21 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:40.345283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 841/1667] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:40.347801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9270ee6-4ded-45c6-ab69-37a470c6ebec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:40.348840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9270ee6-4ded-45c6-ab69-37a470c6ebec service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.349959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:40.350483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9270ee6-4ded-45c6-ab69-37a470c6ebec service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.350936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9270ee6-4ded-45c6-ab69-37a470c6ebec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.373736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4995a2c-5832-41fe-8863-6977a044afe1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:40.377585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9270ee6-4ded-45c6-ab69-37a470c6ebec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:40.379713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9270ee6-4ded-45c6-ab69-37a470c6ebec service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:40.391682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:40.393786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.394784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.395275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] PATH_INFO: `/v3/projects/6deae8a22a664e51b44bb289159e6bd8/users/974ee13c19f64e56b7fe71c0bc36acef/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.411572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 827/1668] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:40.422285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6deae8a22a664e51b44bb289159e6bd8, user_id=974ee13c19f64e56b7fe71c0bc36acef, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:40.425439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:40.518853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 842/1669] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:40 2022] PUT /identity/v3/projects/6deae8a22a664e51b44bb289159e6bd8/users/974ee13c19f64e56b7fe71c0bc36acef/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:40.531395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.531939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.532358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.604172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] MFA Rules not processed for user `974ee13c19f64e56b7fe71c0bc36acef`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:40.608743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:40.793373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 828/1670] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:40.797612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:40.912996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fed46f-8efc-4a96-bef2-f9ee4206b0ed admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:40.927717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:40.928422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:40.928714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:40.928983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:40.929653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:40.931876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:41.120826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 843/1671] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:40 2022] POST /identity/v3/projects => generated 383 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:41.125194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:41.141043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480999f-7773-42b0-b25a-4cce30535774 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:41.331352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:41.341354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:41.341354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:41.341354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:41.341354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:41.341354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:41.604720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:41.606432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 829/1672] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:41 2022] POST /identity/v3/users => generated 298 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:41.628205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15be822-e208-49c9-ae2b-421e5d5ee70c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:41.644568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:41.645215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:41.645506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:41.645786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:41.646463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:41.648185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:41.674744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 844/1673] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:41.682193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:41.703223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d540be2-3a3d-4f4e-8ec5-e1159d2f18ec admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:41.719509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:41.719509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:41.719509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:41.719509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] PATH_INFO: `/v3/projects/80d0980a3c9845a6adcbd373871d9098/users/40639fe8efc0404ebedf5988fd374def/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:41.761717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d0980a3c9845a6adcbd373871d9098, user_id=40639fe8efc0404ebedf5988fd374def, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:41.764214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:41.817816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:41.906514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 830/1674] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:41 2022] PUT /identity/v3/projects/80d0980a3c9845a6adcbd373871d9098/users/40639fe8efc0404ebedf5988fd374def/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:41.911740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:41.931931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0461a6-e8e5-42d5-8282-180746931ff2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:42.109399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:42.110911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.111373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.111989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.112811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:42.114670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:42.133470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 845/1675] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:42.134494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:42.135541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:42.138013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:42.151484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c00f9f-3e7a-499d-a13e-b317eb45fa46 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:42.153748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73513-fb75-4d35-8bd7-1aebe67f543f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:42.160965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:42.161898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.162354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.162815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] PATH_INFO: `/v3/projects/04b4e36569b9430bbfab0c39b6041fdf` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.164383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b4e36569b9430bbfab0c39b6041fdf)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:42.171553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:42.175261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea1b71-225c-4971-99e2-98a186f07cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:42.175939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1b71-225c-4971-99e2-98a186f07cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.176231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1b71-225c-4971-99e2-98a186f07cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.176510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea1b71-225c-4971-99e2-98a186f07cad admin admin] PATH_INFO: `/v3/projects/80d0980a3c9845a6adcbd373871d9098/users/40639fe8efc0404ebedf5988fd374def/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.180910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea1b71-225c-4971-99e2-98a186f07cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d0980a3c9845a6adcbd373871d9098, user_id=40639fe8efc0404ebedf5988fd374def, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:42.185710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea1b71-225c-4971-99e2-98a186f07cad admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:42.209413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 831/1676] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:42 2022] PUT /identity/v3/projects/80d0980a3c9845a6adcbd373871d9098/users/40639fe8efc0404ebedf5988fd374def/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:42.211785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.212179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.213239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.252238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:42.271243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] MFA Rules not processed for user `40639fe8efc0404ebedf5988fd374def`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:42.272508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:42.312016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b4e36569b9430bbfab0c39b6041fdf', 'request_id': 'req-b162734f-542e-4bc4-86c6-8466b141722b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:42.319924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b4e36569b9430bbfab0c39b6041fdf', 'request_id': 'req-b162734f-542e-4bc4-86c6-8466b141722b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:42.430192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b4e36569b9430bbfab0c39b6041fdf', 'request_id': 'req-b162734f-542e-4bc4-86c6-8466b141722b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:42.469372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] The token cache is being invalidate because project 04b4e36569b9430bbfab0c39b6041fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:42.470804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:42.481867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b162734f-542e-4bc4-86c6-8466b141722b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:42.488556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 846/1677] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:42 2022] DELETE /identity/v3/projects/04b4e36569b9430bbfab0c39b6041fdf => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:42.491243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.494482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.495838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.538669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 832/1678] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:42.539632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:42.542915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:42.568721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] MFA Rules not processed for user `e135d9b5894147e6a391d75d8896f094`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:42.571478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:42.667008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eec78c-d454-40d8-a99f-41f7b24c4bd7 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:42.723588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 847/1679] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:42.727005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:42.728828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:42.816010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da3d07-21ef-4bb7-8ab2-631cc4cbb8ea None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:42.821299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408c093-b2a8-40e3-91a8-7c3bb7029558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:42.822089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c093-b2a8-40e3-91a8-7c3bb7029558 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.822437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c093-b2a8-40e3-91a8-7c3bb7029558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.823351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c093-b2a8-40e3-91a8-7c3bb7029558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.886162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:42.886837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:42.887121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:42.887399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] PATH_INFO: `/v3/users/45069d56482443b4ac13fd54d4cfbf1b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:42.895024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45069d56482443b4ac13fd54d4cfbf1b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:42.903799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:42.968834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408c093-b2a8-40e3-91a8-7c3bb7029558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:42.981230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408c093-b2a8-40e3-91a8-7c3bb7029558 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:43.002743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 848/1680] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:43.006635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:43.006966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:43.007253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:43.071319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] MFA Rules not processed for user `e135d9b5894147e6a391d75d8896f094`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:43.073846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:43.138682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 849/1681] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:43.149690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:43.151172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:43.311836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45069d56482443b4ac13fd54d4cfbf1b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:43.318983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45069d56482443b4ac13fd54d4cfbf1b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:43.339924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45069d56482443b4ac13fd54d4cfbf1b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:43.340709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45069d56482443b4ac13fd54d4cfbf1b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:43.380863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45069d56482443b4ac13fd54d4cfbf1b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:43.392495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec862c3b-9241-4bd4-b97b-78f8ad767e93 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:43.419757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:43.421096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:43.421392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:43.421670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:43.432781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:43.451866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:43.455046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:43.501600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 850/1682] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:43.501600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:43.501600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:43.541254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b7a9f-6ce4-4e00-abbc-1829471985c8 service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:43.548865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:43.550102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:43.550562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:43.550981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:43.553407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:43.598785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 833/1683] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:42 2022] DELETE /identity/v3/users/45069d56482443b4ac13fd54d4cfbf1b => generated 0 bytes in 1060 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:43.670807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:43.672369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05918930-d814-4d33-9114-9453d23ae63c service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:43.687708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 851/1684] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:44.474565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:44.890423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:44.893379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:44.903986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5d219-18b0-49b6-8dfe-4c17fd6c99b3 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:44.904235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5d219-18b0-49b6-8dfe-4c17fd6c99b3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:44.904412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5d219-18b0-49b6-8dfe-4c17fd6c99b3 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:44.905822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 852/1685] 10.222.0.65 () {54 vars in 912 bytes} [Mon Aug 29 21:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:44.925520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:44.925865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:44.926157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.140172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:45.146482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:45.223556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ea63a-02c7-47e1-b9cb-2306fc4fe522 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:45.247465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:45.250367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.253410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.254092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] PATH_INFO: `/v3/projects/a84aa5e3024c4adb8b2724daf318900a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.259212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84aa5e3024c4adb8b2724daf318900a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:45.261556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:45.269467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:45.393699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 853/1686] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:45.529137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84aa5e3024c4adb8b2724daf318900a', 'request_id': 'req-05aa0840-a712-424a-9d1e-29c2184f1354'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:45.570171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84aa5e3024c4adb8b2724daf318900a', 'request_id': 'req-05aa0840-a712-424a-9d1e-29c2184f1354'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:45.573794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:45.576354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:45.613871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3606b59-5471-4e68-a130-c4677546a8da None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:45.705659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84aa5e3024c4adb8b2724daf318900a', 'request_id': 'req-05aa0840-a712-424a-9d1e-29c2184f1354'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:45.724592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] The token cache is being invalidate because project a84aa5e3024c4adb8b2724daf318900a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:45.724966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:45.725758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-05aa0840-a712-424a-9d1e-29c2184f1354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:45.728278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 834/1687] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:44 2022] DELETE /identity/v3/projects/a84aa5e3024c4adb8b2724daf318900a => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:45.731031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fae51-70b2-4c8d-8672-c55fb2a978c7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.731515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fae51-70b2-4c8d-8672-c55fb2a978c7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.732022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fae51-70b2-4c8d-8672-c55fb2a978c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.740944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:45.760471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.765799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.771989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.774868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c64fae51-70b2-4c8d-8672-c55fb2a978c7 None None] MFA Rules not processed for user `68c15654f5a14850a1763a020c17c849`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:45.777518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c64fae51-70b2-4c8d-8672-c55fb2a978c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:45.778691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:45.800958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:45.805592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c556a-f507-4c44-8c93-2cb11b28ec7e service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:45.827922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 854/1688] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:45.830278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.832214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.832214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:45.918961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] MFA Rules not processed for user `b7e0a78238e94fa4a8209550c901a9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:45.920541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:45.974912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 835/1689] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:45.980381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378294-9199-4a6c-861a-b82c251afd03 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:45.980645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378294-9199-4a6c-861a-b82c251afd03 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:45.980923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7378294-9199-4a6c-861a-b82c251afd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.026843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 855/1690] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:46.026843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:46.026843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:46.044048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e7378294-9199-4a6c-861a-b82c251afd03 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:46.044137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e7378294-9199-4a6c-861a-b82c251afd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:46.211771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 836/1691] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:46.211771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e4dcb-0c4b-4801-8dcc-297a51cd5e9e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.211771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e4dcb-0c4b-4801-8dcc-297a51cd5e9e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.212738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e4dcb-0c4b-4801-8dcc-297a51cd5e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.244531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2d03a-d4cf-4c08-b30e-0cc59d4058ad None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:46.264967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:46.265582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0d3e4dcb-0c4b-4801-8dcc-297a51cd5e9e None None] MFA Rules not processed for user `b7e0a78238e94fa4a8209550c901a9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:46.266109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.267565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.267955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0d3e4dcb-0c4b-4801-8dcc-297a51cd5e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:46.268239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.276098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 837/1692] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:46.280421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.280421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.280421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.299990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:46.305819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:46.320833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:46.337709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:46.338435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 856/1693] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:46.357101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 838/1694] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:46.362120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:46.374699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b205d4d-a5c4-4ab0-ada3-45de1c76fbdf service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:46.388350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:46.389239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.389683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.389918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.395764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:46.404635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:46.406652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:46.424283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 857/1695] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:46.429317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:46.445622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dec7f-e8ab-4f96-bebf-282643e0ca88 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:46.468582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:46.468582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.468582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.468807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.469539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:46.473817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:46.738496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:46.745486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 839/1696] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:46 2022] POST /identity/v3/projects => generated 385 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:46.750460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:46.768600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09606-245d-4271-8992-595b1f7b3d6f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:46.892222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f3e0-1010-44dd-912a-3452438cf8f6 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:46.902644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:46.907750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.911465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.911968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:46.913138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:46.916975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:46.926068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:46.927191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:46.929046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:46.930264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:47.073016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 840/1697] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:46 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:47.083664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:47.133155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91b47-bc6e-4446-88bd-f57d413b7e6a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:47.151166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:47.153433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:47.155413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:47.156349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:47.162778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:47.163292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:47.166363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:47.169643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:47.208617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 858/1698] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:47.209130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 841/1699] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:47.212280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:47.218656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:47.218891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:47.255259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f9049-7292-499e-80d0-427e9c903328 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:47.277751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:47.278385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:47.281639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:47.282207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:47.283071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:47.288033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:47.323090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 842/1700] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:47.324233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:47.364359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f74df2-b9bd-4ed3-a55b-eaefb6e9efc0 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:47.379556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:47.379810 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:47.380129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:47.380406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] PATH_INFO: `/v3/projects/614b1b46348243d69faf15fe1265b85a/users/1369a1d7570545f5ae8c05081fe6c9f9/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:47.431304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=614b1b46348243d69faf15fe1265b85a, user_id=1369a1d7570545f5ae8c05081fe6c9f9, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:47.434558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:47.445713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c632b2e-b3a1-41ee-8e1e-716e5dd41b38 service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:47.489430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:47.489430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:47.496473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:47.498972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:47.509603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:47.539026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:47.546011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fc8ed-d085-4dbf-9f02-33f58b8692a5 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:47.562889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 859/1701] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:47.623942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 843/1702] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:47 2022] PUT /identity/v3/projects/614b1b46348243d69faf15fe1265b85a/users/1369a1d7570545f5ae8c05081fe6c9f9/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:47.635432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:47.636041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:47.636329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:47.769134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] MFA Rules not processed for user `1369a1d7570545f5ae8c05081fe6c9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:47.770773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:47.913014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:47.914530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:48.096655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 860/1703] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:48.101147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:48.186687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9e539-1523-4c4c-b41c-3cf07bd4e3bf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:48.215284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:48.215993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:48.216279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:48.216557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] PATH_INFO: `/v3/users/fe195a70eb1a470e8f6c7eb1da7dc67c` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:48.218245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe195a70eb1a470e8f6c7eb1da7dc67c)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:48.224280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:48.377759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1bc6d-639b-4b81-9ad2-92bf7604d4a2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:48.391485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58d488-da67-430d-9a40-c366dd970404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:48.392604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58d488-da67-430d-9a40-c366dd970404 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:48.393123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58d488-da67-430d-9a40-c366dd970404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:48.393627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58d488-da67-430d-9a40-c366dd970404 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:48.394581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58d488-da67-430d-9a40-c366dd970404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:48.402390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58d488-da67-430d-9a40-c366dd970404 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:48.647107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 861/1704] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:48 2022] POST /identity/v3/projects => generated 387 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:48.648656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:48.649083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:48.649974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:48.699685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] MFA Rules not processed for user `d3c5edb153624912b87d61dd7ad5e4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:48.701399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:48.778866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe195a70eb1a470e8f6c7eb1da7dc67c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:48.779555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe195a70eb1a470e8f6c7eb1da7dc67c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:48.803296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe195a70eb1a470e8f6c7eb1da7dc67c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:48.804243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe195a70eb1a470e8f6c7eb1da7dc67c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:48.813957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-757c7450-6348-4636-afe9-2b50fbd1cdc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe195a70eb1a470e8f6c7eb1da7dc67c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:48.863624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 862/1705] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:48.863624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:48.863624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:48.899368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 844/1706] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:47 2022] DELETE /identity/v3/users/fe195a70eb1a470e8f6c7eb1da7dc67c => generated 0 bytes in 987 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:48.901870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec42f42-4cca-46fc-a419-1a03b160ff45 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:48.902363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec42f42-4cca-46fc-a419-1a03b160ff45 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:48.902799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec42f42-4cca-46fc-a419-1a03b160ff45 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:48.914899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 845/1707] 10.222.0.65 () {54 vars in 912 bytes} [Mon Aug 29 21:26:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:26:48.914899 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec42f42-4cca-46fc-a419-1a03b160ff45 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:48.918166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2ec42f42-4cca-46fc-a419-1a03b160ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.037615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a476be-0745-40c3-bfa2-748489b3f4f2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:49.053158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:49.185249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec42f42-4cca-46fc-a419-1a03b160ff45 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:49.192493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578506e-b0b4-430f-9b6e-a854cbda3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:49.193109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:49.194230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578506e-b0b4-430f-9b6e-a854cbda3907 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.194438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.194551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578506e-b0b4-430f-9b6e-a854cbda3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.195153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.195570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] PATH_INFO: `/v3/users/974ee13c19f64e56b7fe71c0bc36acef` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.197044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=974ee13c19f64e56b7fe71c0bc36acef)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:49.198573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578506e-b0b4-430f-9b6e-a854cbda3907 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.199540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578506e-b0b4-430f-9b6e-a854cbda3907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:49.199657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:49.215484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578506e-b0b4-430f-9b6e-a854cbda3907 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:49.337739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 863/1708] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:48 2022] POST /identity/v3/users => generated 294 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:49.339213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b50b2-8cd7-48fe-925b-0ab5778d2264 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.339583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b50b2-8cd7-48fe-925b-0ab5778d2264 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.339993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b50b2-8cd7-48fe-925b-0ab5778d2264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.370565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1f8b50b2-8cd7-48fe-925b-0ab5778d2264 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:49.372040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1f8b50b2-8cd7-48fe-925b-0ab5778d2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.380655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 864/1709] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:49.382105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.382390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.382683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.418294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:49.419765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.510286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '974ee13c19f64e56b7fe71c0bc36acef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:49.512870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '974ee13c19f64e56b7fe71c0bc36acef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:49.544905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '974ee13c19f64e56b7fe71c0bc36acef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:49.549123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '974ee13c19f64e56b7fe71c0bc36acef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:49.549235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 865/1710] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:49.550134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:49.551552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.573631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '974ee13c19f64e56b7fe71c0bc36acef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:49.578833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:49.656597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a164e93-fb34-4224-a83c-da2d4443069e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:49.663715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:49.664501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.664804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.665079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.667654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.682572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 846/1711] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:48 2022] DELETE /identity/v3/users/974ee13c19f64e56b7fe71c0bc36acef => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:49.682572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:49.684777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.817571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:49.819556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:49.845276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 866/1712] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:49.846102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:49.852501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.878147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4945-b04f-4125-b9f2-9c20ad84a932 service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:49.880022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f08b4-b047-4c59-a067-e35f0bb3c848 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:49.886403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:49.887904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.888348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.889653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.891646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:49.902475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:49.918219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 847/1713] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:49.923306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:49.926249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:49.984921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:49.986398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:49.986851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:49.987307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:49.991148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:50.035992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6399e75-371c-490e-b7c1-e746e8fbac6e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:50.038790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.040578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103eb1-ceb0-4e77-b295-7592ce333120 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.043818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:50.045421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.045868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.048150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.068430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.070198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.073077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 867/1714] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:50.074371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.074720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.074922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.093368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 848/1715] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:50.094584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:50.109725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:50.113466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:50.126368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9580ae-9d47-42a9-a0e4-dd063b7926c9 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:50.137441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 868/1716] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:50.142715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:50.145932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.146427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.147549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.148555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.157731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.176946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 849/1717] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:50.181174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:50.196974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6316-2078-46d7-83d3-506b75184a9d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:50.212619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858de128-50b8-4986-9fa6-34530d784e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:50.213636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858de128-50b8-4986-9fa6-34530d784e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.214077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858de128-50b8-4986-9fa6-34530d784e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.214510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858de128-50b8-4986-9fa6-34530d784e69 admin admin] PATH_INFO: `/v3/projects/064f4702eac44510b25010d5324954cd/users/d6be37cafda04161b8fdb218df59f971/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.257902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858de128-50b8-4986-9fa6-34530d784e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064f4702eac44510b25010d5324954cd, user_id=d6be37cafda04161b8fdb218df59f971, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.261112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858de128-50b8-4986-9fa6-34530d784e69 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.319818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:50.336419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e050440-bf07-43ae-9a50-a2060844c0e8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:50.350054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:50.351373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.351830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.352290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.382134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.383621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.412803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 869/1718] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:50 2022] PUT /identity/v3/projects/064f4702eac44510b25010d5324954cd/users/d6be37cafda04161b8fdb218df59f971/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:50.421205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.421536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.422268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 850/1719] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:26:50.422435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.441451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:50.443472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:50.472840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ee31a-2a01-432f-8ac1-bc3f3ecc3a74 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:50.499469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] MFA Rules not processed for user `d6be37cafda04161b8fdb218df59f971`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:50.503681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:50.605034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:50.605378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.606538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.606538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] PATH_INFO: `/v3/users/090d9d685305459fb3a72e75f1e597e9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.607305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090d9d685305459fb3a72e75f1e597e9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.609318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.639795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:50.663113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 870/1720] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:50.665462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:50.682374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4b094-300a-4751-8130-adcd3a554c64 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:50.692084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:50.693268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:50.693743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:50.694160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:50.695642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:50.698657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:50.884730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 871/1721] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:50 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:50.889295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:51.020544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090d9d685305459fb3a72e75f1e597e9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.021698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090d9d685305459fb3a72e75f1e597e9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.058189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090d9d685305459fb3a72e75f1e597e9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.062243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090d9d685305459fb3a72e75f1e597e9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.083485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090d9d685305459fb3a72e75f1e597e9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.132857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b1297-657a-4e16-be39-7fd8470b82ae admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:51.148890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be607e-8122-4072-a629-c91049251b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:51.150372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be607e-8122-4072-a629-c91049251b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.150660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be607e-8122-4072-a629-c91049251b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.150933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be607e-8122-4072-a629-c91049251b6e admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.154274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be607e-8122-4072-a629-c91049251b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:51.167362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be607e-8122-4072-a629-c91049251b6e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:51.297180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 851/1722] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:50 2022] DELETE /identity/v3/users/090d9d685305459fb3a72e75f1e597e9 => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:26:51.297180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:51.303166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:51.323073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 872/1723] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:50 2022] POST /identity/v3/users => generated 299 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:51.326746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.327235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.332772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.402210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:51.404299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:51.445975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:51.545537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351b7a-cff0-471b-8c34-5ebd38590374 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:51.586505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:51.587276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.587559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.587835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] PATH_INFO: `/v3/projects/8c7b1ec1f3fa4f38ac616f3d03ec5a10` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.595106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7b1ec1f3fa4f38ac616f3d03ec5a10)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:51.597256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:51.690922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 873/1724] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:51.692046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:51.695283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:51.738898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c48f8-a650-4dc9-98a3-165d71b1a06e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:51.759579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b81f8f-bf25-49b2-a401-2090a8c3c960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:51.760277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b81f8f-bf25-49b2-a401-2090a8c3c960 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.760812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b81f8f-bf25-49b2-a401-2090a8c3c960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.761102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b81f8f-bf25-49b2-a401-2090a8c3c960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.761776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b81f8f-bf25-49b2-a401-2090a8c3c960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:51.765216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b81f8f-bf25-49b2-a401-2090a8c3c960 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:51.789164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 874/1725] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:51.794673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:51.794983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:51.795269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:51.831545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:51.844115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] MFA Rules not processed for user `b7e0a78238e94fa4a8209550c901a9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:51.847916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:51.854655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7b1ec1f3fa4f38ac616f3d03ec5a10', 'request_id': 'req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.874130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7b1ec1f3fa4f38ac616f3d03ec5a10', 'request_id': 'req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:51.930846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 875/1726] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:51.932442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:51.934279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:51.945097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7b1ec1f3fa4f38ac616f3d03ec5a10', 'request_id': 'req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:52.022885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] The token cache is being invalidate because project 8c7b1ec1f3fa4f38ac616f3d03ec5a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:52.023585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:52.030027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:52.040381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 852/1727] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:51 2022] DELETE /identity/v3/projects/8c7b1ec1f3fa4f38ac616f3d03ec5a10 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:52.067754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:52.073912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1ced9-1f24-403d-8117-757581ed6d45 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:52.075752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:52.084347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:52.085057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.086466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.087578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.092574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:52.385518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:52.385719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29326f-d6e9-41a5-89fe-ec429b7f2a5a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:52.386999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9b501-30bd-4859-9425-e7938f561217 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:52.406327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d09e72-5e8d-4da0-852a-5ffe025aa0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:52.408094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09e72-5e8d-4da0-852a-5ffe025aa0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.408881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09e72-5e8d-4da0-852a-5ffe025aa0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.409369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d09e72-5e8d-4da0-852a-5ffe025aa0dd admin admin] PATH_INFO: `/v3/projects/1b34b60ac8f345d49d2360aa89882a8d/users/c89138158656485088bfc6f356248e9e/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.410188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 876/1728] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:52.412838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.412838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.412838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.449087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09e72-5e8d-4da0-852a-5ffe025aa0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b34b60ac8f345d49d2360aa89882a8d, user_id=c89138158656485088bfc6f356248e9e, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:52.450881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:52.454713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:52.458285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d09e72-5e8d-4da0-852a-5ffe025aa0dd admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:52.480008 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 877/1729] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:52.481275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:52.482924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:52.507794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd835040-0774-4899-9c42-b3ff6b1bd9e6 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:52.578076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:52.585724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.592150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.593231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] PATH_INFO: `/v3/projects/6deae8a22a664e51b44bb289159e6bd8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.596287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6deae8a22a664e51b44bb289159e6bd8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:52.599257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:52.623396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 853/1730] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:52 2022] PUT /identity/v3/projects/1b34b60ac8f345d49d2360aa89882a8d/users/c89138158656485088bfc6f356248e9e/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:52.627364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754af219-af6d-498e-99f3-364262fbed86 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.628959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754af219-af6d-498e-99f3-364262fbed86 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.630702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754af219-af6d-498e-99f3-364262fbed86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.689028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-754af219-af6d-498e-99f3-364262fbed86 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:52.690491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-754af219-af6d-498e-99f3-364262fbed86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:52.822328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 854/1731] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:52.823642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754af219-af6d-498e-99f3-364262fbed86 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:52.840819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754af219-af6d-498e-99f3-364262fbed86 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:52.846845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:52.847955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.848406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.848740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.849554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:52.851428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:52.865609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 855/1732] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:52.868991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:52.877889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5f0e-22fb-4326-991b-a96f03f9ea15 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:52.890563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:52.892433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:52.892433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:52.892772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:52.893485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:52.893851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:52.895922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:52.934748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 856/1733] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:52.935699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:52.955210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccad40-fc6f-449e-ba40-4fbcc309eac4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:52.958072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6deae8a22a664e51b44bb289159e6bd8', 'request_id': 'req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:52.974057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6deae8a22a664e51b44bb289159e6bd8', 'request_id': 'req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.052383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6deae8a22a664e51b44bb289159e6bd8', 'request_id': 'req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.084793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] The token cache is being invalidate because project 6deae8a22a664e51b44bb289159e6bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:53.084793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.086375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.095578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 878/1734] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:52 2022] DELETE /identity/v3/projects/6deae8a22a664e51b44bb289159e6bd8 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:53.097060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:53.098899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:53.140901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:53.142376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:53.142805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:53.143255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] PATH_INFO: `/v3/projects/1b34b60ac8f345d49d2360aa89882a8d/users/c89138158656485088bfc6f356248e9e/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:53.147092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b34b60ac8f345d49d2360aa89882a8d, user_id=c89138158656485088bfc6f356248e9e, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:53.158982 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:53.180977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 857/1735] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:52 2022] PUT /identity/v3/projects/1b34b60ac8f345d49d2360aa89882a8d/users/c89138158656485088bfc6f356248e9e/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:53.182203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:53.183190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:53.190042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:53.242653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf2dfc-7862-4a65-9854-2eff9fbd1a8f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:53.384326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940d0f3-6881-43ba-aea7-21ce3db91e8e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:53.387868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24535b28-1b05-47e0-8a40-7c7baba586a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:53.388813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535b28-1b05-47e0-8a40-7c7baba586a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:53.389299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535b28-1b05-47e0-8a40-7c7baba586a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:53.389807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535b28-1b05-47e0-8a40-7c7baba586a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:53.390651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24535b28-1b05-47e0-8a40-7c7baba586a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:53.393402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24535b28-1b05-47e0-8a40-7c7baba586a6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:53.394111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:53.394804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:53.395164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:53.395496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] PATH_INFO: `/v3/users/40639fe8efc0404ebedf5988fd374def` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:53.396763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40639fe8efc0404ebedf5988fd374def)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:53.399026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:53.564648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 879/1736] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:53 2022] POST /identity/v3/projects => generated 373 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:53.570816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7ce0a-48ff-4388-a6c1-0761b9029b91 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:53.571337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7ce0a-48ff-4388-a6c1-0761b9029b91 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:53.571830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7ce0a-48ff-4388-a6c1-0761b9029b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:53.704642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d2a7ce0a-48ff-4388-a6c1-0761b9029b91 None None] MFA Rules not processed for user `c89138158656485088bfc6f356248e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:53.704642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d2a7ce0a-48ff-4388-a6c1-0761b9029b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:53.849365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40639fe8efc0404ebedf5988fd374def'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.854660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40639fe8efc0404ebedf5988fd374def'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.940960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40639fe8efc0404ebedf5988fd374def'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.943096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40639fe8efc0404ebedf5988fd374def'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:53.974853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40639fe8efc0404ebedf5988fd374def'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:54.055201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 880/1737] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:54.062656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:54.064402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:54.064947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:54.111367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] MFA Rules not processed for user `68c15654f5a14850a1763a020c17c849`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:54.123724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:54.126971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 858/1738] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:53 2022] DELETE /identity/v3/users/40639fe8efc0404ebedf5988fd374def => generated 0 bytes in 945 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:54.129006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:54.140389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:54.376943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387fcf79-bf6f-4297-98f7-6414d4fad802 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:54.383491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:54.386242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:54.386242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:54.386242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:54.386242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:54.388804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:54.404160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 881/1739] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:54.406396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:54.412413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:54.463631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:54.547841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 859/1740] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:54 2022] POST /identity/v3/users => generated 287 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:54.548875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:54.551606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:54.580414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd2861-b4f3-4419-97b5-1cf3c1f6552a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:54.593872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:54.595848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:54.596158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:54.597254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] PATH_INFO: `/v3/projects/8f58a8e730ad4824b8ea4d5a18908675` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:54.600528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f58a8e730ad4824b8ea4d5a18908675)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:54.606856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:54.649773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00795-ed75-4000-94bf-b06b195e8874 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:54.726451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:54.760285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf696a-ff94-4b54-9446-affd5709371c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:54.761665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf696a-ff94-4b54-9446-affd5709371c service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:54.761951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf696a-ff94-4b54-9446-affd5709371c service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:54.762228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf696a-ff94-4b54-9446-affd5709371c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:54.863451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f58a8e730ad4824b8ea4d5a18908675', 'request_id': 'req-703bab6a-b029-4f50-828b-58758ee97cef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:54.889431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f58a8e730ad4824b8ea4d5a18908675', 'request_id': 'req-703bab6a-b029-4f50-828b-58758ee97cef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:54.921874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf696a-ff94-4b54-9446-affd5709371c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:54.924598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf696a-ff94-4b54-9446-affd5709371c service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:54.951166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 882/1741] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:54.954794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:54.955823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:54.956691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.006746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f58a8e730ad4824b8ea4d5a18908675', 'request_id': 'req-703bab6a-b029-4f50-828b-58758ee97cef'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.046476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] The token cache is being invalidate because project 8f58a8e730ad4824b8ea4d5a18908675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:55.057643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.060719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.071019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 860/1742] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:54 2022] DELETE /identity/v3/projects/8f58a8e730ad4824b8ea4d5a18908675 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:55.074848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:55.078161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:55.135743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:55.137987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:55.316372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bab6a-b029-4f50-828b-58758ee97cef admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:55.361376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:55.362027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.362310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.362585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.363245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:55.373052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:55.392664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 883/1743] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:55.402240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 861/1744] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:55.418883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:55.419676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:55.421968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:55.439783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ad2d6-df93-4c88-a154-877e58e20211 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:55.453651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806543a7-29e9-44c9-9816-8f0cfbc80594 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:55.457451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:55.458304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.458730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.459155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.460724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:55.463469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:55.464607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.465170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.466296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] PATH_INFO: `/v3/users/118bc888cd5d43c29f2180827502dcc6` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.466893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:55.475160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=118bc888cd5d43c29f2180827502dcc6)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:55.489245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:55.493613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 862/1745] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:55.494534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:55.507065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eac83-47d6-47e4-a73a-5fd610fc2919 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:55.516162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7382c1-eb73-4c4c-97e8-e62ce46d4e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:55.516858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7382c1-eb73-4c4c-97e8-e62ce46d4e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.517288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7382c1-eb73-4c4c-97e8-e62ce46d4e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.517522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7382c1-eb73-4c4c-97e8-e62ce46d4e7d admin admin] PATH_INFO: `/v3/projects/2a9989c25feb4fb78ed6185ac9020be1/users/de00e1be5ed346fabfc545421557cb76/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.559722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7382c1-eb73-4c4c-97e8-e62ce46d4e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9989c25feb4fb78ed6185ac9020be1, user_id=de00e1be5ed346fabfc545421557cb76, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:55.562623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7382c1-eb73-4c4c-97e8-e62ce46d4e7d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:55.659992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 863/1746] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:55 2022] PUT /identity/v3/projects/2a9989c25feb4fb78ed6185ac9020be1/users/de00e1be5ed346fabfc545421557cb76/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:55.666898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:55.667282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:55.667564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:55.739592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118bc888cd5d43c29f2180827502dcc6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.755576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] MFA Rules not processed for user `de00e1be5ed346fabfc545421557cb76`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:55.758771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:55.894963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 864/1747] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:26:55.896287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:55.920048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118bc888cd5d43c29f2180827502dcc6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.941503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118bc888cd5d43c29f2180827502dcc6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.979852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118bc888cd5d43c29f2180827502dcc6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.989758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:55.991690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118bc888cd5d43c29f2180827502dcc6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:55.993858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f794f-401c-49a8-a06d-7991c6999b12 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:56.007025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:56.007868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:56.008323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:56.008614 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:56.009315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:56.012323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:56.015873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 884/1748] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:26:55 2022] DELETE /identity/v3/users/118bc888cd5d43c29f2180827502dcc6 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:56.017737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:56.022574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:56.161130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 865/1749] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:55 2022] POST /identity/v3/projects => generated 373 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:26:56.163392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:56.169166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:56.263091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21468136-4c7f-4c11-af59-84f27a948d61 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:56.280952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:56.442398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:56.443033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:56.443334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:56.443601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:56.455699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:56.580265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd3b6-79fa-4bf4-97f7-7de8ec871634 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:56.607640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:56.609801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:56.611411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:56.612878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:56.615406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:56.618109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:56.660102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:56.661914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420fa6c-e052-4caa-8a88-c4a20f76f71f service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:56.696271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 885/1750] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:26:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:26:56.698320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:56.698777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:56.699659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:56.788068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:56.790230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 866/1751] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:56 2022] POST /identity/v3/users => generated 293 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:56.795306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:56.798085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:56.843128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c508d-b2fd-4ec5-8ae3-1654bd956345 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:56.848869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:56.849606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:56.849952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:56.850296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:56.851011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:56.852937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:56.937748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 867/1752] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:26:56.937748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:56.946712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:57.024953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14790e7c-97b7-400e-a524-f2a5d465d1b5 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:57.045960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:57.046628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.047110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.069753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] PATH_INFO: `/v3/projects/80d0980a3c9845a6adcbd373871d9098` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.073949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d0980a3c9845a6adcbd373871d9098)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:57.088005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:57.100313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 886/1753] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 29 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:57.103602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:57.136422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6720a1-48a6-4698-aa6e-846ee0170963 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:57.143087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:57.145161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.146124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.146931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] PATH_INFO: `/v3/projects/1e43a130143241c2a5df8e5e0b4c10ee/users/503d368ed80c4fd08387c9dd7a906d74/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.181313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e43a130143241c2a5df8e5e0b4c10ee, user_id=503d368ed80c4fd08387c9dd7a906d74, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:57.184439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:57.316352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 887/1754] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:57 2022] PUT /identity/v3/projects/1e43a130143241c2a5df8e5e0b4c10ee/users/503d368ed80c4fd08387c9dd7a906d74/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:57.333309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:57.428359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d0980a3c9845a6adcbd373871d9098', 'request_id': 'req-ddb863f1-7c90-4a63-bb56-8ed637e1afee'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:57.485324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d0980a3c9845a6adcbd373871d9098', 'request_id': 'req-ddb863f1-7c90-4a63-bb56-8ed637e1afee'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:57.626165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089802-f04b-45ef-9451-42326778a7e1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:57.629130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d0980a3c9845a6adcbd373871d9098', 'request_id': 'req-ddb863f1-7c90-4a63-bb56-8ed637e1afee'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:57.653844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] The token cache is being invalidate because project 80d0980a3c9845a6adcbd373871d9098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:57.654230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:57.656302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:57.665398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 868/1755] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:56 2022] DELETE /identity/v3/projects/80d0980a3c9845a6adcbd373871d9098 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:57.666677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:57.668966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:57.779663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:57.780152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.780371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.780610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.781410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:57.783094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:57.796401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 888/1756] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:26:57.799396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:57.804503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:57.832577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb863f1-7c90-4a63-bb56-8ed637e1afee admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:57.838253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:57.839514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.839928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.840160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.842553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:57.903055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:57.903055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95061728-1d14-42c2-8956-af256c399b6c service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:57.921676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d8450-a4d0-4bda-b3ed-74f97d785f1c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:57.924015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 869/1757] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:57.925731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.926074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.931866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.932188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:57.932188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:57.932188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:57.932188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:57.932498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.000172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:58.002207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55644042-ca3a-44e5-a04f-9b471a81df6e service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:58.017969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] MFA Rules not processed for user `d3c5edb153624912b87d61dd7ad5e4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:58.022823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.031378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 889/1758] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:58.045481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.051739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.064068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.091979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:58.146220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 870/1759] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:58.146220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:58.149473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:58.151459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.155045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.184492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 890/1760] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:58.184492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:58.194309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.221085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb348ea2-efff-4410-8939-5d08d576df5e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:58.227257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b718b-c9fe-4a74-aaa2-b16768cb66d8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:58.240597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:58.243863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.244262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ad23c-8a70-42ba-ae4f-c1f3d116909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:58.245399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.245692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] PATH_INFO: `/v3/projects/bba0123e9f4c474793c1532073a08fb0` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.247563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ad23c-8a70-42ba-ae4f-c1f3d116909f admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.247930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ad23c-8a70-42ba-ae4f-c1f3d116909f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.256145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ad23c-8a70-42ba-ae4f-c1f3d116909f admin admin] PATH_INFO: `/v3/projects/1e43a130143241c2a5df8e5e0b4c10ee/users/503d368ed80c4fd08387c9dd7a906d74/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.256353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba0123e9f4c474793c1532073a08fb0)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:58.257421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ad23c-8a70-42ba-ae4f-c1f3d116909f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e43a130143241c2a5df8e5e0b4c10ee, user_id=503d368ed80c4fd08387c9dd7a906d74, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:58.259150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:58.261577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ad23c-8a70-42ba-ae4f-c1f3d116909f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:58.315975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 871/1761] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:26:58 2022] PUT /identity/v3/projects/1e43a130143241c2a5df8e5e0b4c10ee/users/503d368ed80c4fd08387c9dd7a906d74/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:58.315975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce5d7c-014a-48a4-86f0-5c3ffda18a22 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.315975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce5d7c-014a-48a4-86f0-5c3ffda18a22 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.315975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce5d7c-014a-48a4-86f0-5c3ffda18a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.369399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-25ce5d7c-014a-48a4-86f0-5c3ffda18a22 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:58.372751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-25ce5d7c-014a-48a4-86f0-5c3ffda18a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.498648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba0123e9f4c474793c1532073a08fb0', 'request_id': 'req-84a09f13-25fa-49da-b68a-f5912014eda1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:58.512145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba0123e9f4c474793c1532073a08fb0', 'request_id': 'req-84a09f13-25fa-49da-b68a-f5912014eda1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:58.533861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 872/1762] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:58.535251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc752b-2252-4c73-bcb1-0c70021a35cc None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.535558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc752b-2252-4c73-bcb1-0c70021a35cc None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.536066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc752b-2252-4c73-bcb1-0c70021a35cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.577933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9efc752b-2252-4c73-bcb1-0c70021a35cc None None] MFA Rules not processed for user `503d368ed80c4fd08387c9dd7a906d74`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:58.579470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9efc752b-2252-4c73-bcb1-0c70021a35cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.681076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba0123e9f4c474793c1532073a08fb0', 'request_id': 'req-84a09f13-25fa-49da-b68a-f5912014eda1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:58.701717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] The token cache is being invalidate because project bba0123e9f4c474793c1532073a08fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:26:58.704524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:58.705319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:26:58.708049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 891/1763] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:26:58 2022] DELETE /identity/v3/projects/bba0123e9f4c474793c1532073a08fb0 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:26:58.710539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:58.714092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.756935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 873/1764] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:58.758430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.758742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.761026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.817590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:58.819169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.940447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a09f13-25fa-49da-b68a-f5912014eda1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:58.940447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:58.940447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:58.940447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:58.941051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:58.941051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:58.941051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:58.941051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:58.956173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 874/1765] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:58.960089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 892/1766] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:58.960536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:58.960536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:58.963244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:26:58.968294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:58.980448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc35e77-8293-496b-91fc-a8473af11073 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:59.004752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:59.005435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:59.005741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:59.006025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:59.007680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:59.010332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:59.061530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45217d-a23e-4e28-a5bd-b87c25e064ec None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:59.072842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c141d-4be1-4d98-8c10-3a3404784e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:59.073525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c141d-4be1-4d98-8c10-3a3404784e71 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:59.073812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c141d-4be1-4d98-8c10-3a3404784e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:59.074139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c141d-4be1-4d98-8c10-3a3404784e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:59.087565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c141d-4be1-4d98-8c10-3a3404784e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:59.089863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c141d-4be1-4d98-8c10-3a3404784e71 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:59.155131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 875/1767] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:26:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:26:59.157369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:59.159720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:59.160395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:59.229645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:26:59.229645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:26:59.229645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 876/1768] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:26:59.247489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:59.268837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc3146-2b72-4108-aba7-2162187d7045 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:59.276485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:59.277442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:59.277822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:59.278197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:59.279306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:26:59.279306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:59.282913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:59.299293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 877/1769] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:26:59.304600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 893/1770] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:58 2022] POST /identity/v3/projects => generated 409 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:59.304809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:59.307570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:59.334248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ef7de-8bf9-4fdd-90ca-bf834371a28e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:59.557840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:59.558636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:59.566201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:59.566573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:59.567317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:59.571918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:59.585627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca85dcf-6c59-42f5-8119-54a054f6f184 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:59.602485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:26:59.603211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:26:59.603447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:26:59.603695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:26:59.609733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:26:59.623014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:26:59.774205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 894/1771] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:59 2022] POST /identity/v3/users => generated 305 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:26:59.778716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:26:59.800280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b151-8a9c-466d-9cbf-f541e1e90d34 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:26:59.805330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 878/1772] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:26:59 2022] POST /identity/v3/projects => generated 369 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:26:59.806881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.003005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.003930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.004384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.004821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.005679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.007648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.037733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 895/1773] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:00.048698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.084794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb27a8c-148a-4509-aa74-23e3d2fcfd20 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.119070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d69d3-99e4-4757-9e6d-889e3136d52a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.123156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.123815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.132608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.133088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.135619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.137689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.152260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.152260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.152260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.152260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.152752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.155583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 896/1774] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:00.156515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.156939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.182083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a40d68-c78a-4178-a65e-33b2d6491046 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.207535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d10b2-963a-443b-9fa5-8be8137bdccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.207535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d10b2-963a-443b-9fa5-8be8137bdccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.207535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d10b2-963a-443b-9fa5-8be8137bdccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.207535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d10b2-963a-443b-9fa5-8be8137bdccb admin admin] PATH_INFO: `/v3/projects/e2784f5faf9d4c9f817dd797c6ea2ab5/users/9b27625c6c9a4cff9eba21909522e017/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.257005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d10b2-963a-443b-9fa5-8be8137bdccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2784f5faf9d4c9f817dd797c6ea2ab5, user_id=9b27625c6c9a4cff9eba21909522e017, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.259993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069d10b2-963a-443b-9fa5-8be8137bdccb admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.361424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 879/1775] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:26:59 2022] POST /identity/v3/users => generated 285 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:00.362643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.400559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6487b11-48ce-439c-8492-f2be1e2ec4a3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.421682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.423451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.424075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.424360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.426212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.431713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.436253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 897/1776] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:00 2022] PUT /identity/v3/projects/e2784f5faf9d4c9f817dd797c6ea2ab5/users/9b27625c6c9a4cff9eba21909522e017/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:00.437796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.438778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.439048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.490480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 880/1777] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:00.492024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.546199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27afce-d465-412e-80ee-b957fb2791ee admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.576468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] MFA Rules not processed for user `9b27625c6c9a4cff9eba21909522e017`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:00.580716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:00.753831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.754521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.754853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.755126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.755781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.760290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.811028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 881/1778] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:00.812015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.834123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 898/1779] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:00.836994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac89ea-8f36-4725-9e3a-a90d7dc4d8bb admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.840425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:00.855626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.856590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.856877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.857156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] PATH_INFO: `/v3/projects/3bb99dd0b9844836a50f7dce291eb2d2/users/e53081d78f4d44088859aebdb9e7d329/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.862046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d8bd3-201c-4322-9bfb-3695da7bb36a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:00.882600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d79d62-9953-4acc-bed6-2c8f564aa01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:00.883259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79d62-9953-4acc-bed6-2c8f564aa01a admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:00.883544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79d62-9953-4acc-bed6-2c8f564aa01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:00.883823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d79d62-9953-4acc-bed6-2c8f564aa01a admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:00.884576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79d62-9953-4acc-bed6-2c8f564aa01a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.886595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d79d62-9953-4acc-bed6-2c8f564aa01a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:00.906626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bb99dd0b9844836a50f7dce291eb2d2, user_id=e53081d78f4d44088859aebdb9e7d329, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:00.909288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:01.049355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 882/1780] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:00 2022] PUT /identity/v3/projects/3bb99dd0b9844836a50f7dce291eb2d2/users/e53081d78f4d44088859aebdb9e7d329/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:01.057416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 899/1781] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:00 2022] POST /identity/v3/projects => generated 407 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:01.058091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:01.062429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.063366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.064055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.203154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:01.217057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] MFA Rules not processed for user `e53081d78f4d44088859aebdb9e7d329`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:01.228526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:01.314769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c8e2d-adb9-4dec-ad26-021f1476dbfd admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:01.447020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 900/1782] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:01.454438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:01.563498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:01.564094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.568096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.568937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.570748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:01.593073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:01.594907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c9ea-2259-4d8f-b351-e1ff3c0d46d8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:01.607239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:01.608691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:01.608980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:01.609268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:01.610530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:01.615154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.200244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 901/1783] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:01 2022] POST /identity/v3/projects => generated 367 bytes in 747 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:02.202103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:02.202684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 883/1784] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:01 2022] POST /identity/v3/users => generated 310 bytes in 1145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:27:02.203583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:02.206915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:02.223002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca06c6-8830-4d0f-80c1-62214ac8fe1d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:02.374161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:02.387487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.387487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.387487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.387487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:02.387487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.410232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f2aa4-db32-44c7-9040-708a840df50f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:02.417120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:02.418071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.418517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.418961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] PATH_INFO: `/v3/users/de00e1be5ed346fabfc545421557cb76` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.420829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de00e1be5ed346fabfc545421557cb76)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:02.426639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.530580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 884/1785] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:02 2022] POST /identity/v3/users => generated 290 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:02.532233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:02.594358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527963c-9476-4f1c-badd-ff024359beaf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:02.595392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd96158-2985-4197-a6e6-c4fff2850c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:02.600676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd96158-2985-4197-a6e6-c4fff2850c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.600676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd96158-2985-4197-a6e6-c4fff2850c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.600676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd96158-2985-4197-a6e6-c4fff2850c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.600676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd96158-2985-4197-a6e6-c4fff2850c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:02.600676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd96158-2985-4197-a6e6-c4fff2850c74 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.636766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 885/1786] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:02.644512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59a347-0254-419b-8fdf-47a876a9540b None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.645113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59a347-0254-419b-8fdf-47a876a9540b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.645807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59a347-0254-419b-8fdf-47a876a9540b None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.648157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 886/1787] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:27:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:02.649405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59a347-0254-419b-8fdf-47a876a9540b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:02.686321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59a347-0254-419b-8fdf-47a876a9540b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:02.698451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:02.699356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.699786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.700244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.701237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:02.705637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.750140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 887/1788] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:02.751758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:02.774746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0d3eb-3e29-4fe8-a4dd-135d3fc83039 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:02.784729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c1739-52dc-4e9e-bf57-b66c6e818784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:02.785439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c1739-52dc-4e9e-bf57-b66c6e818784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.785726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c1739-52dc-4e9e-bf57-b66c6e818784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.786007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c1739-52dc-4e9e-bf57-b66c6e818784 admin admin] PATH_INFO: `/v3/projects/2bde6509bbae45b9a7affa8ef7081ba4/users/31c10fd77ada4e408dc3efccca437a4d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:02.834159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c1739-52dc-4e9e-bf57-b66c6e818784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bde6509bbae45b9a7affa8ef7081ba4, user_id=31c10fd77ada4e408dc3efccca437a4d, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:02.834159 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c1739-52dc-4e9e-bf57-b66c6e818784 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:02.997666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 888/1789] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:02 2022] PUT /identity/v3/projects/2bde6509bbae45b9a7affa8ef7081ba4/users/31c10fd77ada4e408dc3efccca437a4d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:02.998966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:02.999343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:02.999738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:03.028004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de00e1be5ed346fabfc545421557cb76'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:03.077669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:03.077669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:03.173876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 889/1790] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:27:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:03.174820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:03.189484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de00e1be5ed346fabfc545421557cb76'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:03.209884 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de00e1be5ed346fabfc545421557cb76'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:03.241252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de00e1be5ed346fabfc545421557cb76'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:03.242404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de00e1be5ed346fabfc545421557cb76'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:03.250615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 902/1791] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:02 2022] DELETE /identity/v3/users/de00e1be5ed346fabfc545421557cb76 => generated 0 bytes in 1049 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:03.258196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:03.258196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:03.355265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415f6e-9932-408c-93f3-ce8384eff3c1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:03.420968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0857be6-5958-4fb9-ae3d-46a15bfb1df9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:03.427301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:03.428487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:03.428977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:03.429443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:03.430305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:03.432326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:03.449616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 903/1792] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:03.450786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:03.461996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:03.496352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:03.497677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:03.497995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:03.498306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] PATH_INFO: `/v3/projects/6dba6472081d422f96b6558628b9151d/users/bcee956025464651a7cacf52e00b32a2/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:03.523427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dba6472081d422f96b6558628b9151d, user_id=bcee956025464651a7cacf52e00b32a2, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:03.527013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:03.627634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214296a-46a1-4c1f-8ce5-6f43b967eb7a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:03.646044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:03.647137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:03.647643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:03.648175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:03.651835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:03.667479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:03.669241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2231905-85d9-41cc-95e9-a6bd68804404 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:03.692145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 904/1793] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:03.698118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:03.698118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:03.698118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:03.757669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:03.759022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:03.778142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 905/1794] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:03.784649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:03.804171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0d13-f453-4a56-ac98-0e27ced0a13c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:03.824470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:03.825783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:03.826139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:03.826664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] PATH_INFO: `/v3/projects/2bde6509bbae45b9a7affa8ef7081ba4/users/31c10fd77ada4e408dc3efccca437a4d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:03.844064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bde6509bbae45b9a7affa8ef7081ba4, user_id=31c10fd77ada4e408dc3efccca437a4d, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:03.847490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:03.855578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 890/1795] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:03 2022] PUT /identity/v3/projects/6dba6472081d422f96b6558628b9151d/users/bcee956025464651a7cacf52e00b32a2/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:03.855578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:03.871972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:03.894391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 906/1796] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:03 2022] PUT /identity/v3/projects/2bde6509bbae45b9a7affa8ef7081ba4/users/31c10fd77ada4e408dc3efccca437a4d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:03.895525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:03.897458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:04.026989 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835eb4-4c4d-40f6-a8d2-539dce7770fb admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:04.035707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4363c4-c1f5-4a76-81bd-c9e90ec01f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:04.035707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4363c4-c1f5-4a76-81bd-c9e90ec01f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.035707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4363c4-c1f5-4a76-81bd-c9e90ec01f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.035707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4363c4-c1f5-4a76-81bd-c9e90ec01f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.036385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4363c4-c1f5-4a76-81bd-c9e90ec01f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:04.037012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afa510-e8ce-4cc3-a2b5-e5d2fa0c5aa6 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:04.037408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4363c4-c1f5-4a76-81bd-c9e90ec01f9d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:04.063138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 907/1797] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:04.065224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.065686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.065964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.143680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] MFA Rules not processed for user `31c10fd77ada4e408dc3efccca437a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:04.147556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:04.299253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:04.299974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.302564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.302852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.319465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:04.321532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:04.340478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 891/1798] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:04.341782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:04.355922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6ddf0-1be8-4490-8a0e-a1185b6f456c service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:04.362570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5312d-525e-4e05-b54b-611c62700cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:04.363285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5312d-525e-4e05-b54b-611c62700cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.363569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5312d-525e-4e05-b54b-611c62700cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.363925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5312d-525e-4e05-b54b-611c62700cbd admin admin] PATH_INFO: `/v3/projects/6dba6472081d422f96b6558628b9151d/users/bcee956025464651a7cacf52e00b32a2/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.366331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5312d-525e-4e05-b54b-611c62700cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dba6472081d422f96b6558628b9151d, user_id=bcee956025464651a7cacf52e00b32a2, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:04.369838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5312d-525e-4e05-b54b-611c62700cbd admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:04.379345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 908/1799] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:04.388585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:04.392404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:04.410384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 892/1800] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:04 2022] PUT /identity/v3/projects/6dba6472081d422f96b6558628b9151d/users/bcee956025464651a7cacf52e00b32a2/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:04.422432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.423993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.424193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.467214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] MFA Rules not processed for user `bcee956025464651a7cacf52e00b32a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:04.468475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:04.510376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f99e31-0c4a-4b69-85ac-3867896854d6 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:04.582984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 893/1801] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:04.599517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:04.605226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:04.608548 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:04.609198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.609455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.610412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.621083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2023fb-270f-4a21-b4c0-31aa1c5693b6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:04.626225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:04.627433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:04.628153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:04.628800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:04.644799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:04.647589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:04.685705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 894/1802] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:04.737059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:04.738580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:04.788084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 909/1803] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:04 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:05.072157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:05.557720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:05.559487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:05.681103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac26fb5-7f7a-4f0f-8294-460c4b2608ad service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:05.687516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:05.689112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:05.689618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:05.690104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] PATH_INFO: `/v3/projects/2a9989c25feb4fb78ed6185ac9020be1` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:05.691531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9989c25feb4fb78ed6185ac9020be1)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:05.693759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:05.816376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:05.827624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e50824-9a6b-46c5-ba18-ab74428d1526 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:05.834658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:05.836619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:05.836619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:05.836619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:05.838375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:05.962257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:05.962553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9989c25feb4fb78ed6185ac9020be1', 'request_id': 'req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:05.964131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:05.986835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9989c25feb4fb78ed6185ac9020be1', 'request_id': 'req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:06.003110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 910/1804] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:05 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:06.097201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9989c25feb4fb78ed6185ac9020be1', 'request_id': 'req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:06.117780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] The token cache is being invalidate because project 2a9989c25feb4fb78ed6185ac9020be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:06.117780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:06.119536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:06.121400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 895/1805] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:05 2022] DELETE /identity/v3/projects/2a9989c25feb4fb78ed6185ac9020be1 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:06.126136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:06.143415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171dd35-9ee4-4a76-9538-348dd9ced7a2 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:06.306347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:06.306996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:06.307284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:06.307657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] PATH_INFO: `/v3/users/503d368ed80c4fd08387c9dd7a906d74` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:06.309656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503d368ed80c4fd08387c9dd7a906d74)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:06.317756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:07.623350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:07.626038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:07.672731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503d368ed80c4fd08387c9dd7a906d74'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:07.756471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57a40-b388-4fc7-93f3-4c4f1a521eac admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:07.760892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:07.761644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:07.761992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:07.762329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:07.764527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:07.845966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503d368ed80c4fd08387c9dd7a906d74'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:07.866206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503d368ed80c4fd08387c9dd7a906d74'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:07.885632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503d368ed80c4fd08387c9dd7a906d74'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:07.890822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503d368ed80c4fd08387c9dd7a906d74'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:07.903555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 911/1806] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:06 2022] DELETE /identity/v3/users/503d368ed80c4fd08387c9dd7a906d74 => generated 0 bytes in 1779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:07.904235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:07.905734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3949446-59d1-49e3-8fd9-4cdc582b7c50 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:07.932244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 896/1807] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:08.165468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:10.075788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:10.075788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:10.196154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f3a05-f178-442e-9ea2-1c07de270376 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:10.202691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:10.204736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.205262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.205788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] PATH_INFO: `/v3/projects/1e43a130143241c2a5df8e5e0b4c10ee` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.208359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e43a130143241c2a5df8e5e0b4c10ee)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:10.211346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:10.407543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e43a130143241c2a5df8e5e0b4c10ee', 'request_id': 'req-0d8e245c-b672-4fd2-a386-da4d2848d4be'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:10.413411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e43a130143241c2a5df8e5e0b4c10ee', 'request_id': 'req-0d8e245c-b672-4fd2-a386-da4d2848d4be'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:10.516987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e43a130143241c2a5df8e5e0b4c10ee', 'request_id': 'req-0d8e245c-b672-4fd2-a386-da4d2848d4be'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:10.555312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] The token cache is being invalidate because project 1e43a130143241c2a5df8e5e0b4c10ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:10.555795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:10.557134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0d8e245c-b672-4fd2-a386-da4d2848d4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:10.561134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 912/1808] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:10 2022] DELETE /identity/v3/projects/1e43a130143241c2a5df8e5e0b4c10ee => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:10.566825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.566825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.567592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.598788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:10.600274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:10.698479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 897/1809] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:10.700575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.701159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.701905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.726333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:10.727836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:10.743707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 913/1810] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:10.744833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:10.760754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a18db-f14d-4af4-bd29-a7eceb12e800 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:10.766564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:10.767198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.767481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.767786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.768547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:10.768858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:10.771420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aec6ee-074f-43d1-a51f-ae5d7cc14a40 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:10.786239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 898/1811] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:10.791641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:10.806771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2474e0-1a23-428d-b053-bdfd9a4f7941 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:10.831661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:10.832606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.832961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.833880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.834958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:10.836509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fbc9-1ad3-4e79-85ec-fc65c2e05730 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.836831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fbc9-1ad3-4e79-85ec-fc65c2e05730 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.837114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fbc9-1ad3-4e79-85ec-fc65c2e05730 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.841269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:10.848027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 899/1812] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:27:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:10.868888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:10.869532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:10.869945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:10.903342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:10.904815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:11.006970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 900/1813] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:11.015353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:11.032493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.032629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 914/1814] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:10 2022] POST /identity/v3/projects => generated 355 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:11.035718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:11.036498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.072250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26eb5b-cdb7-4367-be82-fe04fff85716 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.173325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0e9f-b78a-434e-ba43-c3a8f8df5f83 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.180589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.181209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.181763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.181937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.183050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.186285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.208962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.208962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.209842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.209842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.219981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.222108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.238519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 901/1815] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:11.372492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 915/1816] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:11 2022] POST /identity/v3/users => generated 278 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:11.377234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.401953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bb79a-872a-47c5-a2e1-fe46b8a1d55c service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.415068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.416076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.416523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.416990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.418089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.431378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.458733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 902/1817] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:11.463418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.478037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd671404-88f6-4858-945e-c01a0d01e242 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.483839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e40eec-f4eb-4bf8-a81e-44d1e14b1df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.484973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e40eec-f4eb-4bf8-a81e-44d1e14b1df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.485553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e40eec-f4eb-4bf8-a81e-44d1e14b1df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.486029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e40eec-f4eb-4bf8-a81e-44d1e14b1df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.487174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e40eec-f4eb-4bf8-a81e-44d1e14b1df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.489457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e40eec-f4eb-4bf8-a81e-44d1e14b1df3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.501626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 916/1818] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:11.506674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.517729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab82f10-4eeb-431d-9260-1d9219aa6480 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.522456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.523112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.523391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.523727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] PATH_INFO: `/v3/projects/7fa895f89a774e4caa29cfaf819b0666/users/25d0bce60985440cac97099ebd1a0f6c/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.541326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fa895f89a774e4caa29cfaf819b0666, user_id=25d0bce60985440cac97099ebd1a0f6c, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.544382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.635653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 903/1819] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:11 2022] PUT /identity/v3/projects/7fa895f89a774e4caa29cfaf819b0666/users/25d0bce60985440cac97099ebd1a0f6c/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:11.647667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.648110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.648517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.679812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] MFA Rules not processed for user `25d0bce60985440cac97099ebd1a0f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:11.681160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:11.755382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 917/1820] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:11.758728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:11.838299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81c27b-dd65-4a00-b1d9-15b1f66cf532 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:11.847726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:11.847726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:11.847726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:11.847726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:11.848338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:11.850461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:11.973799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 904/1821] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:11 2022] POST /identity/v3/projects => generated 357 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:11.989275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.002442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1028fc-0fb7-43e8-85a8-4c9ef41bd754 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.084951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.085815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.086103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.086373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.087014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.089058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.212166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 918/1822] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:11 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:12.212985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.228020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fdaa5-0dca-4d01-b4b4-a758d8af585b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.231754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.232449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.232780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.233011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.233773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.235404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.246940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 905/1823] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:12.251322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.262778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40914f-bef7-4a20-9231-ee792728f518 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.268537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.269353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.269632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.269909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] PATH_INFO: `/v3/projects/f068fc1a514a4be2bbb295fba3f8a2d9/users/92ea3ea2790a4e31a0c8485ad4090bc5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.288711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f068fc1a514a4be2bbb295fba3f8a2d9, user_id=92ea3ea2790a4e31a0c8485ad4090bc5, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.301376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.383436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 919/1824] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:12 2022] PUT /identity/v3/projects/f068fc1a514a4be2bbb295fba3f8a2d9/users/92ea3ea2790a4e31a0c8485ad4090bc5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:12.387621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.399204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b559857-631e-4d15-80fc-ba5036abd677 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.457146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b46e9-3674-415d-b97d-d2cbc04aed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.457763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b46e9-3674-415d-b97d-d2cbc04aed71 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.458041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b46e9-3674-415d-b97d-d2cbc04aed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.458313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b46e9-3674-415d-b97d-d2cbc04aed71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.458953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b46e9-3674-415d-b97d-d2cbc04aed71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.460850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b46e9-3674-415d-b97d-d2cbc04aed71 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.471569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 906/1825] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:12.476268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.490587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5a3ef-bcf6-4401-8635-dac95b95ab1a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.491616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.492315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.492686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.492881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] PATH_INFO: `/v3/projects/f068fc1a514a4be2bbb295fba3f8a2d9/users/92ea3ea2790a4e31a0c8485ad4090bc5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.494866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f068fc1a514a4be2bbb295fba3f8a2d9, user_id=92ea3ea2790a4e31a0c8485ad4090bc5, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.497625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.512293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 920/1826] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:12 2022] PUT /identity/v3/projects/f068fc1a514a4be2bbb295fba3f8a2d9/users/92ea3ea2790a4e31a0c8485ad4090bc5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:12.525372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.525372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.525372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.558736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] MFA Rules not processed for user `92ea3ea2790a4e31a0c8485ad4090bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:12.559995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:12.635346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 907/1827] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:12.649077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.651866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:12.686636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:12.692481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:12.721970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69561edf-7e06-49eb-93a0-50e6e2b1b7ed admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.725897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.726550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.726833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.727111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.741422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.743070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.754155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 921/1828] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:12.791157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280ad5f-3095-48b2-9cd6-eaef4d22d836 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:12.791157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:12.791157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:12.791157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:12.791471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] PATH_INFO: `/v3/users/e53081d78f4d44088859aebdb9e7d329` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:12.791471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53081d78f4d44088859aebdb9e7d329)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:12.791471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:12.990167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:13.038411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53081d78f4d44088859aebdb9e7d329'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:13.039124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53081d78f4d44088859aebdb9e7d329'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:13.040375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:13.059945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0bbd2-e407-4a45-a120-e6cd8c46c6d6 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:13.061078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53081d78f4d44088859aebdb9e7d329'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:13.062032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53081d78f4d44088859aebdb9e7d329'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:13.066585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:13.067260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:13.067549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:13.067842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:13.069838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:13.071910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53081d78f4d44088859aebdb9e7d329'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:13.158492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:13.159995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e8b5f-fae4-46c0-9b93-0e288bd6ba8d service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:13.183278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 922/1829] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:13.186030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 908/1830] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:12 2022] DELETE /identity/v3/users/e53081d78f4d44088859aebdb9e7d329 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:13.191628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:13.192178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:13.192574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:13.214186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:13.215461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:13.278843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 923/1831] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:13.289735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:13.291267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:13.350589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fba47d-c787-4035-b035-c1691896b8d3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:13.354397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:13.355019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:13.358814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:13.358814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:13.365994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:13.367611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:13.381024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 909/1832] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:14.129052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:14.129052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:14.135707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7216-fa31-4930-82ac-ae3c0e6f79d2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:14.140829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:14.141647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:14.142031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:14.142557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] PATH_INFO: `/v3/projects/3bb99dd0b9844836a50f7dce291eb2d2` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:14.143913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bb99dd0b9844836a50f7dce291eb2d2)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:14.145941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:14.377245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bb99dd0b9844836a50f7dce291eb2d2', 'request_id': 'req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.386038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bb99dd0b9844836a50f7dce291eb2d2', 'request_id': 'req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.565797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bb99dd0b9844836a50f7dce291eb2d2', 'request_id': 'req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.578941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] The token cache is being invalidate because project 3bb99dd0b9844836a50f7dce291eb2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:14.579939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.579939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.581519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 924/1833] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:14 2022] DELETE /identity/v3/projects/3bb99dd0b9844836a50f7dce291eb2d2 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:14.585561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:14.596418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c0f5c-e320-4bb5-ba22-24fddef1abf3 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:14.654091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:14.654641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:14.654919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:14.655189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] PATH_INFO: `/v3/users/bcee956025464651a7cacf52e00b32a2` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:14.657030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcee956025464651a7cacf52e00b32a2)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:14.658994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:14.971908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcee956025464651a7cacf52e00b32a2'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.972656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcee956025464651a7cacf52e00b32a2'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.988054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcee956025464651a7cacf52e00b32a2'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.988901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcee956025464651a7cacf52e00b32a2'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:14.997428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcee956025464651a7cacf52e00b32a2'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:15.087519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 910/1834] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:14 2022] DELETE /identity/v3/users/bcee956025464651a7cacf52e00b32a2 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:15.184829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:16.440937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:16.561848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:16.561848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:16.574380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9401-67cc-4134-9eb9-a6835ce36fd4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:16.664721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:16.674507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:16.674507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:16.674507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] PATH_INFO: `/v3/projects/6dba6472081d422f96b6558628b9151d` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:16.674507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dba6472081d422f96b6558628b9151d)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:16.679723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:16.785121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:16.789908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:16.815470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b831bcb-aeb0-4e60-b086-8488933dfa91 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:16.820916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dba6472081d422f96b6558628b9151d', 'request_id': 'req-420a8e6f-8ec7-466b-bf38-e6a457c81443'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:16.839172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dba6472081d422f96b6558628b9151d', 'request_id': 'req-420a8e6f-8ec7-466b-bf38-e6a457c81443'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:16.899297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:16.900689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:16.901083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:16.901459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] PATH_INFO: `/v3/users/1369a1d7570545f5ae8c05081fe6c9f9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:16.904348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1369a1d7570545f5ae8c05081fe6c9f9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:16.909145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:16.972524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dba6472081d422f96b6558628b9151d', 'request_id': 'req-420a8e6f-8ec7-466b-bf38-e6a457c81443'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:16.990601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] The token cache is being invalidate because project 6dba6472081d422f96b6558628b9151d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:16.990946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:16.991793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-420a8e6f-8ec7-466b-bf38-e6a457c81443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:16.993386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 925/1835] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:16 2022] DELETE /identity/v3/projects/6dba6472081d422f96b6558628b9151d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:16.999033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028304-29b5-4753-ad58-77f5d0d3a6ef None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:16.999326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028304-29b5-4753-ad58-77f5d0d3a6ef None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:16.999660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028304-29b5-4753-ad58-77f5d0d3a6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.031783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-92028304-29b5-4753-ad58-77f5d0d3a6ef None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:17.033327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-92028304-29b5-4753-ad58-77f5d0d3a6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.043834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 926/1836] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:17.059177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.059535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.059816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.086598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:17.088090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.100938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 927/1837] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:17.107633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.136614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9fd29-9c82-4d1d-af55-07e1316d65ac None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.142951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.144636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.152476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.153213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.154378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:17.154947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.157337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.171952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 928/1838] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:17.175784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.190469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6e0ae-82bf-4436-8488-db5847d0bc60 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.197098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.197847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.198252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.198626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.199850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.202340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.292578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1369a1d7570545f5ae8c05081fe6c9f9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:17.293640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1369a1d7570545f5ae8c05081fe6c9f9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:17.320073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1369a1d7570545f5ae8c05081fe6c9f9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:17.320847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1369a1d7570545f5ae8c05081fe6c9f9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:17.327801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-67e7dc4c-3204-4231-9070-17891039826f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1369a1d7570545f5ae8c05081fe6c9f9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:17.468870 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 929/1839] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:17 2022] POST /identity/v3/projects => generated 375 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:17.473750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.475490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.540898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 911/1840] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:16 2022] DELETE /identity/v3/users/1369a1d7570545f5ae8c05081fe6c9f9 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:17.544782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9389-0532-4e9e-9ed4-1afc7a287416 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.545582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.549061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.549535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.575920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:17.577384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.615238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.615917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.616574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.616862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.617812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.620095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.661540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 912/1841] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:17.684635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.684635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:17.740905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7668e6-25e7-45ef-a1df-ff44b6368e79 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.744604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.745223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.745503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.745773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.756167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.757629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.768567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 913/1842] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:17.779578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 930/1843] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:17 2022] POST /identity/v3/users => generated 288 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:17.784854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.797298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368080a-b7c6-4c58-adc7-78306d1c426c service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.803526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.804464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.804849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.805387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.806089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.807919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.820403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 914/1844] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:17.824225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.837444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c299e93-9d34-4447-80b7-166c1a3bb03c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.843748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84b516-5104-499b-84d9-1eca06140795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.844479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84b516-5104-499b-84d9-1eca06140795 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.844757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84b516-5104-499b-84d9-1eca06140795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.845031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84b516-5104-499b-84d9-1eca06140795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.845679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84b516-5104-499b-84d9-1eca06140795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.847340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84b516-5104-499b-84d9-1eca06140795 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:17.866080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 931/1845] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:17.872133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:17.878583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c1298-516d-4ad6-b414-d669cd0a9566 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:17.883521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:17.884343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:17.884627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:17.884903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] PATH_INFO: `/v3/projects/871bc0e9f9424aa4add3f3f4db17b121/users/f1ee0b947bd840ddb838ad243ede51ed/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:17.910092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=871bc0e9f9424aa4add3f3f4db17b121, user_id=f1ee0b947bd840ddb838ad243ede51ed, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:17.913329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:18.100464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 915/1846] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:17 2022] PUT /identity/v3/projects/871bc0e9f9424aa4add3f3f4db17b121/users/f1ee0b947bd840ddb838ad243ede51ed/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:18.113002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.113317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.113648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.154269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] MFA Rules not processed for user `f1ee0b947bd840ddb838ad243ede51ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:18.157056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:18.248414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 932/1847] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:18.253803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:18.333092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a998-fb64-4357-a146-57255adca59c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:18.343693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa9760-fe69-414a-b034-4c163f95a2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:18.344907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa9760-fe69-414a-b034-4c163f95a2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.345266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa9760-fe69-414a-b034-4c163f95a2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.345761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa9760-fe69-414a-b034-4c163f95a2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.347347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa9760-fe69-414a-b034-4c163f95a2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:18.351189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa9760-fe69-414a-b034-4c163f95a2e6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:18.408605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:18.410943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:18.446826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb4fbe-cb3a-4a46-890f-6d1c4b18b20c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:18.452743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:18.453563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.454332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.454889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] PATH_INFO: `/v3/projects/614b1b46348243d69faf15fe1265b85a` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.456795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=614b1b46348243d69faf15fe1265b85a)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:18.461238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:18.493048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 916/1848] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:18 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:18.503694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fe10c-0fdc-4e34-a5dc-4c389c25ecc3 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.504798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fe10c-0fdc-4e34-a5dc-4c389c25ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.505356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fe10c-0fdc-4e34-a5dc-4c389c25ecc3 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.513883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 917/1849] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:27:18 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:27:18.515053 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fe10c-0fdc-4e34-a5dc-4c389c25ecc3 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:18.615598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fe10c-0fdc-4e34-a5dc-4c389c25ecc3 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:18.622531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45325358-5b25-4c49-9aed-3a25e6bbb57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:18.623027 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45325358-5b25-4c49-9aed-3a25e6bbb57a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.623506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45325358-5b25-4c49-9aed-3a25e6bbb57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.623865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45325358-5b25-4c49-9aed-3a25e6bbb57a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.624475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45325358-5b25-4c49-9aed-3a25e6bbb57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:18.626826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45325358-5b25-4c49-9aed-3a25e6bbb57a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:18.722059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614b1b46348243d69faf15fe1265b85a', 'request_id': 'req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:18.730673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614b1b46348243d69faf15fe1265b85a', 'request_id': 'req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:18.785732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 918/1850] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:18 2022] POST /identity/v3/users => generated 295 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:18.789210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.789482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.789759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.811278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:18.812866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:18.879636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614b1b46348243d69faf15fe1265b85a', 'request_id': 'req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:18.892172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 919/1851] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:27:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:18.892835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:18.896147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] The token cache is being invalidate because project 614b1b46348243d69faf15fe1265b85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:18.896631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:18.897688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:18.900262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 933/1852] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:18 2022] DELETE /identity/v3/projects/614b1b46348243d69faf15fe1265b85a => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:18.904068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:18.906998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:18.978055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728ff1f-c955-427e-8285-4439ed3a0b72 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:18.984234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:18.984916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:18.985258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:18.986235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:18.988366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:18.990500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.005284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 920/1853] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:19.006547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:19.009783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.028979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1793dd-9670-4620-9c95-3576b28de6ba admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:19.037837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704c1fd-1349-42dd-9aac-0bf2df7c3d2d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:19.039468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:19.040350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.040645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.041064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.051674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:19.052381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.052663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.052939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] PATH_INFO: `/v3/users/d6be37cafda04161b8fdb218df59f971` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.054388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6be37cafda04161b8fdb218df59f971)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:19.056821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.057492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.080835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:19.082545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.096055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 921/1854] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:19.097414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:19.098787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.114322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a066-d4ad-498e-86a5-26ed387b2dca service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:19.119362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:19.120112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.120398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.120817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] PATH_INFO: `/v3/projects/7ef98d38db134c77a7ed576ebfdcc355/users/e6f34552a8934d1482cfcec0c7c715ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.145873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef98d38db134c77a7ed576ebfdcc355, user_id=e6f34552a8934d1482cfcec0c7c715ae, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:19.150017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.309094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 922/1855] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:19 2022] PUT /identity/v3/projects/7ef98d38db134c77a7ed576ebfdcc355/users/e6f34552a8934d1482cfcec0c7c715ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:19.319611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:19.432555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6be37cafda04161b8fdb218df59f971'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:19.455067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a393f-b54f-4958-bf27-8867f2a877b5 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:19.463083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:19.464203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.464579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.465129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.465797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:19.467549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.483352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 923/1856] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:19.495771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:19.507529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99d545-b490-48db-ae44-4b85c36dea5f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:19.514990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177075b-0f06-4321-9deb-419a6f761e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:19.515716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177075b-0f06-4321-9deb-419a6f761e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.516238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177075b-0f06-4321-9deb-419a6f761e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.516626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177075b-0f06-4321-9deb-419a6f761e6c admin admin] PATH_INFO: `/v3/projects/7ef98d38db134c77a7ed576ebfdcc355/users/e6f34552a8934d1482cfcec0c7c715ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.519636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177075b-0f06-4321-9deb-419a6f761e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef98d38db134c77a7ed576ebfdcc355, user_id=e6f34552a8934d1482cfcec0c7c715ae, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:19.522656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177075b-0f06-4321-9deb-419a6f761e6c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.533782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6be37cafda04161b8fdb218df59f971'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:19.548753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6be37cafda04161b8fdb218df59f971'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:19.548929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 924/1857] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:19 2022] PUT /identity/v3/projects/7ef98d38db134c77a7ed576ebfdcc355/users/e6f34552a8934d1482cfcec0c7c715ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:19.557898 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.558171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.558471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.569030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6be37cafda04161b8fdb218df59f971'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:19.573563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6be37cafda04161b8fdb218df59f971'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:19.591043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] MFA Rules not processed for user `e6f34552a8934d1482cfcec0c7c715ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:19.592369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.602726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 934/1858] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:18 2022] DELETE /identity/v3/users/d6be37cafda04161b8fdb218df59f971 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:19.723269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 925/1859] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:19.806053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:19.809252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.888431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a122e-b92e-4046-87cd-d976b1abcfee admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:19.894106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:19.894791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:19.895049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:19.895552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:19.897961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.910755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:19.912459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:19.973996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:19.975661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:19.986984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 935/1860] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:20.016748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9ec9c-c6b5-4208-832c-b02dde59e3c0 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:20.152074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:20.153155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.153441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.153716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] PATH_INFO: `/v3/users/9b27625c6c9a4cff9eba21909522e017` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.155117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b27625c6c9a4cff9eba21909522e017)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:20.157247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:20.372712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:20.379927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:20.381313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:20.403398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e3e22-a5ee-4c79-9304-e6ac8e0d7e39 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:20.409558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:20.411426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.411426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.411426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] PATH_INFO: `/v3/projects/064f4702eac44510b25010d5324954cd` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.412464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=064f4702eac44510b25010d5324954cd)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:20.414612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:20.473028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b27625c6c9a4cff9eba21909522e017'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.474355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b27625c6c9a4cff9eba21909522e017'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.504726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b27625c6c9a4cff9eba21909522e017'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.505484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b27625c6c9a4cff9eba21909522e017'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.513307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ffbd611-114e-4cc7-bf54-f4b9b7837a10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b27625c6c9a4cff9eba21909522e017'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.607583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064f4702eac44510b25010d5324954cd', 'request_id': 'req-e130d494-38b6-41c2-87bb-8e73219a9a46'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.613921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064f4702eac44510b25010d5324954cd', 'request_id': 'req-e130d494-38b6-41c2-87bb-8e73219a9a46'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.662245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 926/1861] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:19 2022] DELETE /identity/v3/users/9b27625c6c9a4cff9eba21909522e017 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:20.668171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.668461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.668740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.707436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:20.708887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:20.754220 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064f4702eac44510b25010d5324954cd', 'request_id': 'req-e130d494-38b6-41c2-87bb-8e73219a9a46'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.767760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] The token cache is being invalidate because project 064f4702eac44510b25010d5324954cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:20.768464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.769355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:20.775104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 936/1862] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:20 2022] DELETE /identity/v3/projects/064f4702eac44510b25010d5324954cd => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:20.776690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:20.781839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 927/1863] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:20.782921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:20.789064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:20.791000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:20.874337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9a40-6014-4f29-8af5-e5a9d4d6cfb6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:20.878955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:20.879609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.879923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.880207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.894676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d494-38b6-41c2-87bb-8e73219a9a46 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:20.897353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:20.898842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:20.911970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 928/1864] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:20.914639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:20.915304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:20.915575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:20.915853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] PATH_INFO: `/v3/users/c89138158656485088bfc6f356248e9e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:20.917232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89138158656485088bfc6f356248e9e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:20.919549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:21.239998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89138158656485088bfc6f356248e9e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:21.331362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89138158656485088bfc6f356248e9e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:21.341021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89138158656485088bfc6f356248e9e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:21.357379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89138158656485088bfc6f356248e9e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:21.358192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89138158656485088bfc6f356248e9e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:21.370208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 937/1865] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:20 2022] DELETE /identity/v3/users/c89138158656485088bfc6f356248e9e => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:21.482256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:21.877085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:21.879662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:21.895225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeca037-194a-41a7-bca5-d5ef7ad03d99 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:21.928556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:21.928556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:21.986134 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:21.987537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:21.987819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:21.988127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] PATH_INFO: `/v3/projects/e2784f5faf9d4c9f817dd797c6ea2ab5` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:21.989428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2784f5faf9d4c9f817dd797c6ea2ab5)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:21.991684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:22.038759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63624b3-55c3-498b-b907-4b7d131254fc admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:22.044838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:22.045562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.045844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.046119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] PATH_INFO: `/v3/projects/1b34b60ac8f345d49d2360aa89882a8d` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.047763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b34b60ac8f345d49d2360aa89882a8d)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:22.050291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:22.130516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2784f5faf9d4c9f817dd797c6ea2ab5', 'request_id': 'req-855d0b21-7d2d-4937-b731-e4ec2a851de5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.135764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2784f5faf9d4c9f817dd797c6ea2ab5', 'request_id': 'req-855d0b21-7d2d-4937-b731-e4ec2a851de5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.205648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2784f5faf9d4c9f817dd797c6ea2ab5', 'request_id': 'req-855d0b21-7d2d-4937-b731-e4ec2a851de5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.223297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] The token cache is being invalidate because project e2784f5faf9d4c9f817dd797c6ea2ab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:22.223493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.224411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.229776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 929/1866] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:21 2022] DELETE /identity/v3/projects/e2784f5faf9d4c9f817dd797c6ea2ab5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:22.230764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:22.232962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:22.234852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b34b60ac8f345d49d2360aa89882a8d', 'request_id': 'req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.250713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b34b60ac8f345d49d2360aa89882a8d', 'request_id': 'req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.348541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d0b21-7d2d-4937-b731-e4ec2a851de5 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:22.360657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b34b60ac8f345d49d2360aa89882a8d', 'request_id': 'req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.387056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] The token cache is being invalidate because project 1b34b60ac8f345d49d2360aa89882a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:22.394832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.394832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c498b95a-68e5-44b6-9e1f-07b16d8a02a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.394832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 938/1867] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:21 2022] DELETE /identity/v3/projects/1b34b60ac8f345d49d2360aa89882a8d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:22.405706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b8c56-e191-45a4-9f44-f9cd2dfd7dcd None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.406993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b8c56-e191-45a4-9f44-f9cd2dfd7dcd None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.407515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b8c56-e191-45a4-9f44-f9cd2dfd7dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.462813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:22.463475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.463763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.464074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] PATH_INFO: `/v3/users/31c10fd77ada4e408dc3efccca437a4d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.466022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c10fd77ada4e408dc3efccca437a4d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:22.468369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:22.469328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-7a8b8c56-e191-45a4-9f44-f9cd2dfd7dcd None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:22.470839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7a8b8c56-e191-45a4-9f44-f9cd2dfd7dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:22.501838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 939/1868] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:22.511595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.512381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.512882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.546328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:22.547849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:22.564762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 940/1869] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:22.569139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:22.580434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a59c4-f654-45dd-91c6-e3f65d189782 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:22.585447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:22.586337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.586618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.586947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.587602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:22.587957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:22.589619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:22.604128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 941/1870] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:27:22.608487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:22.621086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a70f5-6b4e-4075-a37f-c869e1dd5a27 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:22.626778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:22.627445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.627936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.628436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.629296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:22.631467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:22.813828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c10fd77ada4e408dc3efccca437a4d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.814948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c10fd77ada4e408dc3efccca437a4d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.845892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c10fd77ada4e408dc3efccca437a4d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.849566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c10fd77ada4e408dc3efccca437a4d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.850768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 942/1871] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:22 2022] POST /identity/v3/projects => generated 367 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:22.855717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:22.857770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:22.871970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c10fd77ada4e408dc3efccca437a4d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:22.950584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b6c09d-24b4-4c22-9d50-72cf07e954b0 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:22.963649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:22.964436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:22.964802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:22.970106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:22.970106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:22.970106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:23.000834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 930/1872] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:22 2022] DELETE /identity/v3/users/31c10fd77ada4e408dc3efccca437a4d => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:23.023638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:23.104063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 943/1873] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:22 2022] POST /identity/v3/users => generated 284 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:23.107906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:23.179527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06b449-be4a-4e66-85dc-d57df1d95b73 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:23.192377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:23.192973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.193617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.193617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.194287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:23.195969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:23.212133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 931/1874] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:23.213888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:23.231191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57141731-ef01-4c6e-8ebe-98c8db691a6a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:23.243417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa5073-7abc-47ba-a60e-03d610dcdb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:23.244208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5073-7abc-47ba-a60e-03d610dcdb72 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.244535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5073-7abc-47ba-a60e-03d610dcdb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.244819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa5073-7abc-47ba-a60e-03d610dcdb72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.245590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa5073-7abc-47ba-a60e-03d610dcdb72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:23.247254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa5073-7abc-47ba-a60e-03d610dcdb72 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:23.262139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 944/1875] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:23.267442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:23.283780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8c1fb-82a7-437c-9ecd-6a8a91ca99a4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:23.284291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:23.284969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.285242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.285779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] PATH_INFO: `/v3/projects/f6ccc70ac12444558aa2e6f4b606573a/users/870103e49ddd46c28fb4dff1b24d41f6/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.306006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ccc70ac12444558aa2e6f4b606573a, user_id=870103e49ddd46c28fb4dff1b24d41f6, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:23.309341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:23.532235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 932/1876] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:23 2022] PUT /identity/v3/projects/f6ccc70ac12444558aa2e6f4b606573a/users/870103e49ddd46c28fb4dff1b24d41f6/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:23.547804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.548368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.548807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.563918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:23.567961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:23.585134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] MFA Rules not processed for user `870103e49ddd46c28fb4dff1b24d41f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:23.586647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:23.663441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 945/1877] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:23.677540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:23.679015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:23.684545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07568eff-2812-4bbd-b90e-23b61e706e9a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:23.691514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:23.692204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.692486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.692929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] PATH_INFO: `/v3/projects/2bde6509bbae45b9a7affa8ef7081ba4` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.694238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bde6509bbae45b9a7affa8ef7081ba4)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:23.697116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:23.805967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153fa1-b59b-4479-9a62-108837158b6d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:23.810620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a56ac9-1aeb-43c8-b69c-964c35f99d88 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:23.811278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a56ac9-1aeb-43c8-b69c-964c35f99d88 service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.811562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a56ac9-1aeb-43c8-b69c-964c35f99d88 service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.811839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a56ac9-1aeb-43c8-b69c-964c35f99d88 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:23.822315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a56ac9-1aeb-43c8-b69c-964c35f99d88 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:23.824231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a56ac9-1aeb-43c8-b69c-964c35f99d88 service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:23.840245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 946/1878] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:23.864777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bde6509bbae45b9a7affa8ef7081ba4', 'request_id': 'req-03e35333-58a5-47a4-93ec-a853fd0efb57'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:23.869474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bde6509bbae45b9a7affa8ef7081ba4', 'request_id': 'req-03e35333-58a5-47a4-93ec-a853fd0efb57'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:23.959593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bde6509bbae45b9a7affa8ef7081ba4', 'request_id': 'req-03e35333-58a5-47a4-93ec-a853fd0efb57'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:23.972587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] The token cache is being invalidate because project 2bde6509bbae45b9a7affa8ef7081ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:23.972874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:23.973816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-03e35333-58a5-47a4-93ec-a853fd0efb57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:23.979656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 933/1879] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:23 2022] DELETE /identity/v3/projects/2bde6509bbae45b9a7affa8ef7081ba4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:23.982343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:23.983137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:23.983446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.006006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:24.007316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:24.090872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 947/1880] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:24.099278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.099776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.100650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.123382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:24.128072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:24.136733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 934/1881] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:24.142248 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.153091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59e840-3762-4f5e-9211-8325ab783e36 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.160370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.162221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:24.168532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.169148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.169635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.170397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:24.171358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.173854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.187213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 948/1882] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:24.197806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.207587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878c2c-9be4-4a3d-a5ce-a6bb040e6089 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.212477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.213078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.213355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.213630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.214264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.216204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.373121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 935/1883] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:24 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:24.377529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.387699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ced4-36e0-4aeb-9c40-210d043e34b2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.446987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.447866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.448179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.448460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.449100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.451079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.588449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 949/1884] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:24 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:24.589256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.600033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230fccf-b637-46af-8c2c-1d909064f3ee admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.605130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.605816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.606203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.606572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.607360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.609056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.619875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 936/1885] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:24.627750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.642828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25158a3b-9e1f-4abc-a3fe-0e59cc5c328e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.649014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacd7c7-d04a-4145-a04f-5da0b29224ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.649904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd7c7-d04a-4145-a04f-5da0b29224ba admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.650182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd7c7-d04a-4145-a04f-5da0b29224ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.650456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacd7c7-d04a-4145-a04f-5da0b29224ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.651105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacd7c7-d04a-4145-a04f-5da0b29224ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.653763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacd7c7-d04a-4145-a04f-5da0b29224ba admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.667815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 950/1886] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:24.672319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:24.684056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954a6e5-42a5-42ef-b7d6-d149ce5a5c38 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:24.689794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:24.690651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.691024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.691402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] PATH_INFO: `/v3/projects/82ae5d3eee6d4687b85cce1034b63291/users/e2a10566f44546a99cad926581ce91b7/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.713287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ae5d3eee6d4687b85cce1034b63291, user_id=e2a10566f44546a99cad926581ce91b7, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:24.716094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:24.857595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 937/1887] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:24 2022] PUT /identity/v3/projects/82ae5d3eee6d4687b85cce1034b63291/users/e2a10566f44546a99cad926581ce91b7/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:24.868513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:24.868952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:24.869332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:24.904760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] MFA Rules not processed for user `e2a10566f44546a99cad926581ce91b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:24.907169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:24.972490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:24.990829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 951/1888] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:24.996176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:25.123725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c82a44-9a00-4210-b09d-b8399350f114 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:25.123725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:25.123725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:25.123725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:25.124910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:25.124910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:25.126637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:25.472896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 938/1889] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:24 2022] POST /identity/v3/projects => generated 381 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:25.477641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:25.489981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609baad-ad2f-46b9-93fa-9ea131f54af4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:25.548345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:25.548969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:25.549252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:25.549529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:25.550214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:25.552296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:25.652354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 952/1890] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:25 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:25.656597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:25.667590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398b2ef-c718-43f8-8b0b-f1130ff57087 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:25.672564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:25.673289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:25.673568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:25.673839 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:25.674479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:25.676136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:25.687189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 939/1891] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:25.693610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:25.704030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34405ed1-c87c-48ee-a325-b887e2b7b1b8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:25.708582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:25.709240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:25.709519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:25.709792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] PATH_INFO: `/v3/projects/2693867333b642af825e1330f123cd52/users/54a701ac73804ce6afdf16e48f80092c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:25.725707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2693867333b642af825e1330f123cd52, user_id=54a701ac73804ce6afdf16e48f80092c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:25.728398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:25.893283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 953/1892] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:25 2022] PUT /identity/v3/projects/2693867333b642af825e1330f123cd52/users/54a701ac73804ce6afdf16e48f80092c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:25.900578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:25.912687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c06678-2947-4e40-b408-05e17a469cf4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:25.987434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c3f6f-822b-41cb-a949-e01c28d02cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:25.988802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c3f6f-822b-41cb-a949-e01c28d02cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:25.988802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c3f6f-822b-41cb-a949-e01c28d02cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:25.988976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c3f6f-822b-41cb-a949-e01c28d02cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:25.989832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c3f6f-822b-41cb-a949-e01c28d02cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:25.993183 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c3f6f-822b-41cb-a949-e01c28d02cf1 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:26.007957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 940/1893] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:26.016234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:26.036778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec51879-3aa8-4160-bd6c-46735d8516b9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:26.045468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:26.046509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.046899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.047239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] PATH_INFO: `/v3/projects/2693867333b642af825e1330f123cd52/users/54a701ac73804ce6afdf16e48f80092c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.049707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2693867333b642af825e1330f123cd52, user_id=54a701ac73804ce6afdf16e48f80092c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:26.052837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:26.070676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 954/1894] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:26 2022] PUT /identity/v3/projects/2693867333b642af825e1330f123cd52/users/54a701ac73804ce6afdf16e48f80092c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:26.092434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.092842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.093214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.124550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] MFA Rules not processed for user `54a701ac73804ce6afdf16e48f80092c`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:26.127012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:26.221353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 941/1895] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:26.233526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:26.235940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:26.299571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd728800-2af5-4f12-9420-dce322483df4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:26.301650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:26.302263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:26.302544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:26.302816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:26.311970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:26.313940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:26.324025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 955/1896] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:27.080342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:27.256335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff81da9-f305-42ce-b564-cd4842b2fbfb None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:27.271294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:27.272037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:27.272611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:27.273024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] PATH_INFO: `/v3/users/e2a10566f44546a99cad926581ce91b7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:27.278262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a10566f44546a99cad926581ce91b7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:27.278262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:27.329682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:27.331779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.345608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53d663-2949-45e2-9e3e-3d4a31e0ca02 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:27.349617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:27.350269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:27.351114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:27.351546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:27.354087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.446660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:27.448166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd1ccc-868f-461a-b4ea-a594cc0aefb7 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:27.463769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 956/1897] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:27.574702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a10566f44546a99cad926581ce91b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:27.575428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a10566f44546a99cad926581ce91b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:27.589870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a10566f44546a99cad926581ce91b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:27.590607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a10566f44546a99cad926581ce91b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:27.610016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a10566f44546a99cad926581ce91b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:27.712599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:27.717406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 942/1898] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:27 2022] DELETE /identity/v3/users/e2a10566f44546a99cad926581ce91b7 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:27.723917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:27.723917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:27.723917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:27.749260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:27.751014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.815046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 957/1899] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:27.825109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:27.826824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:27.901948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b855504-9656-4d1b-8486-e07af5ef372f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:27.906829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:27.908229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:27.908597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:27.908973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:27.919578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:27.921956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:27.935413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 943/1900] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:28.105316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:28.581693 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:28.583084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:28.595827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab850811-120c-4110-9db0-3ea9877eef4e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:28.601410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:28.603142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:28.603442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:28.603918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] PATH_INFO: `/v3/projects/82ae5d3eee6d4687b85cce1034b63291` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:28.605702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ae5d3eee6d4687b85cce1034b63291)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:28.611407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:28.768109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ae5d3eee6d4687b85cce1034b63291', 'request_id': 'req-544ec8f9-0145-42fb-9e0b-f60c5674f078'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:28.773629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ae5d3eee6d4687b85cce1034b63291', 'request_id': 'req-544ec8f9-0145-42fb-9e0b-f60c5674f078'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:28.880021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ae5d3eee6d4687b85cce1034b63291', 'request_id': 'req-544ec8f9-0145-42fb-9e0b-f60c5674f078'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:28.892836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] The token cache is being invalidate because project 82ae5d3eee6d4687b85cce1034b63291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:28.893128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:28.893804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:28.895351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 958/1901] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:28 2022] DELETE /identity/v3/projects/82ae5d3eee6d4687b85cce1034b63291 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:28.899376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:28.909567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab61e02-f642-423d-b7b1-1231ceb9ab91 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:28.968279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:28.969764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:28.969885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:28.970206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] PATH_INFO: `/v3/users/54a701ac73804ce6afdf16e48f80092c` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:28.971475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a701ac73804ce6afdf16e48f80092c)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:28.973452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:29.312731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a701ac73804ce6afdf16e48f80092c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:29.313333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a701ac73804ce6afdf16e48f80092c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:29.327004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a701ac73804ce6afdf16e48f80092c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:29.327732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a701ac73804ce6afdf16e48f80092c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:29.336448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-439ec903-e1e6-41a2-86b6-59a5c0a59af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a701ac73804ce6afdf16e48f80092c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:29.466158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 944/1902] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:28 2022] DELETE /identity/v3/users/54a701ac73804ce6afdf16e48f80092c => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:30.168042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:30.169483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:30.191654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ec8f9-0145-42fb-9e0b-f60c5674f078 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:30.253379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:30.254031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:30.254327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:30.254605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] PATH_INFO: `/v3/projects/2693867333b642af825e1330f123cd52` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:30.255823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2693867333b642af825e1330f123cd52)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:30.257804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:30.495821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2693867333b642af825e1330f123cd52', 'request_id': 'req-01d3de05-c419-4e19-8c44-902a783dd164'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:30.503399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2693867333b642af825e1330f123cd52', 'request_id': 'req-01d3de05-c419-4e19-8c44-902a783dd164'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:30.629159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2693867333b642af825e1330f123cd52', 'request_id': 'req-01d3de05-c419-4e19-8c44-902a783dd164'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:30.636523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] The token cache is being invalidate because project 2693867333b642af825e1330f123cd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:30.636815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:30.637469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-01d3de05-c419-4e19-8c44-902a783dd164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:30.640087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 959/1903] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:30 2022] DELETE /identity/v3/projects/2693867333b642af825e1330f123cd52 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:30.646005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:30.646473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:30.646548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:30.667058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:30.668307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:30.731742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 945/1904] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:30.740594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:30.741294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:30.741840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:30.754860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:30.767183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:30.768493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:30.777007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 960/1905] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:30.781408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:30.791994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ffcb4-dacf-4463-bbb4-06be67105688 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:30.796880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:30.797500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:30.797777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:30.798052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:30.798731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:30.799029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:30.800690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:30.813024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 946/1906] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:30.817911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:30.828682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7cb58-084c-4e85-b02a-d07aa4ae87a5 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:30.833588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:30.834239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:30.834595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:30.834872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:30.835682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:30.838078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:31.036214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 961/1907] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:30 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:31.040879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:31.053354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af152a-7bac-4771-82f3-9d2810dd906a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:31.113164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:31.113783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.114064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.114339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.114986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:31.116978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:31.283763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 947/1908] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:31 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:31.285307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:31.295614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e63af-5b69-446e-9316-a168c95cc555 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:31.301191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:31.301844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.302132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.302410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.303064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:31.304778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:31.316119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 962/1909] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:31.320048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:31.330238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e16a4-1b4d-4a19-9f68-e7856db7e1c9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:31.335003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b18a37-ee36-466e-97d7-0b69cb9ee2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:31.335624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18a37-ee36-466e-97d7-0b69cb9ee2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.335933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18a37-ee36-466e-97d7-0b69cb9ee2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.336565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18a37-ee36-466e-97d7-0b69cb9ee2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.342182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18a37-ee36-466e-97d7-0b69cb9ee2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:31.342182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18a37-ee36-466e-97d7-0b69cb9ee2ca admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:31.350245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 948/1910] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:31.354309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:31.365684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce11c17-306e-4a15-bdc3-12df36fff755 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:31.370666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:31.371339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.371622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.375005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] PATH_INFO: `/v3/projects/c3a396cb6ec74c9c9df40b29d4b84c1e/users/b897d61b7aae4a62998a6cc469880a24/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.388281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a396cb6ec74c9c9df40b29d4b84c1e, user_id=b897d61b7aae4a62998a6cc469880a24, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:31.390962 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:31.474972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 963/1911] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:31 2022] PUT /identity/v3/projects/c3a396cb6ec74c9c9df40b29d4b84c1e/users/b897d61b7aae4a62998a6cc469880a24/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:31.486814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:31.488621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.488955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.489230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.517704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] MFA Rules not processed for user `b897d61b7aae4a62998a6cc469880a24`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:31.518830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:31.584552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 949/1912] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:31.589243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:31.654531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d780ef1-9067-4e0b-a8e4-d0d4dc3b4ae4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:31.660411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:31.661628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.661628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.661886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.662752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:31.671496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:31.799363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 964/1913] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:31 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:31.800503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:31.811451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ca912-2be2-4751-b795-26ea03af0d1e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:31.868133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:31.868756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:31.869038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:31.869315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:31.869966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:31.871971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:32.046411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 950/1914] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:31 2022] POST /identity/v3/users => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:32.050960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:32.068437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84fcf6-c310-43a2-b56a-3ba871abbcd9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:32.073848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:32.074469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.074760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.075032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.075684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:32.077330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:32.091744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 965/1915] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:32.095446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:32.097652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:32.113131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cec219-7443-475d-899d-531556dacffc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:32.119698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:32.120645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.120943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.121360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] PATH_INFO: `/v3/projects/e645c9435a0e41f1a066e9edeee6ff5c/users/9d192bf2abb449c1b1c0e89b8827f928/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.147106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e645c9435a0e41f1a066e9edeee6ff5c, user_id=9d192bf2abb449c1b1c0e89b8827f928, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:32.149978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:32.274661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 951/1916] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:32 2022] PUT /identity/v3/projects/e645c9435a0e41f1a066e9edeee6ff5c/users/9d192bf2abb449c1b1c0e89b8827f928/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:32.278874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:32.289299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af513441-10c2-4f04-abb1-0646a0422c87 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:32.352790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad7c7c-fb62-42ae-ac65-9b7242ef5d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:32.353402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad7c7c-fb62-42ae-ac65-9b7242ef5d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.353684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad7c7c-fb62-42ae-ac65-9b7242ef5d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.353957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad7c7c-fb62-42ae-ac65-9b7242ef5d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.354597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad7c7c-fb62-42ae-ac65-9b7242ef5d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:32.356238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad7c7c-fb62-42ae-ac65-9b7242ef5d93 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:32.368039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 966/1917] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:32.371788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:32.382938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03f1e1-3ba6-4956-a975-bc3034a39c1b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:32.387615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:32.390440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.390440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.390440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] PATH_INFO: `/v3/projects/e645c9435a0e41f1a066e9edeee6ff5c/users/9d192bf2abb449c1b1c0e89b8827f928/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.390803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e645c9435a0e41f1a066e9edeee6ff5c, user_id=9d192bf2abb449c1b1c0e89b8827f928, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:32.393575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:32.404324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 952/1918] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:32 2022] PUT /identity/v3/projects/e645c9435a0e41f1a066e9edeee6ff5c/users/9d192bf2abb449c1b1c0e89b8827f928/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:32.414777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.415063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.415487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.446301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] MFA Rules not processed for user `9d192bf2abb449c1b1c0e89b8827f928`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:32.447579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:32.520487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 967/1919] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:32.531484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:32.533195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:32.595425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916c41d-9474-4f3e-82e0-772f8f27a73c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:32.599754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f43bdf-8682-4671-b61d-9b29d3e199de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:32.600418 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43bdf-8682-4671-b61d-9b29d3e199de service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:32.600703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43bdf-8682-4671-b61d-9b29d3e199de service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:32.601011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43bdf-8682-4671-b61d-9b29d3e199de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:32.673071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f43bdf-8682-4671-b61d-9b29d3e199de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:32.674499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f43bdf-8682-4671-b61d-9b29d3e199de service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:32.686435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 953/1920] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:33.320111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:34.091749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:35.865804 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:36.528201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:37.759697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:37.761491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:37.852402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c0952-5f37-4c13-8e67-2e7de3e91a77 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:37.853929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:37.854615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:37.854930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:37.855210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] PATH_INFO: `/v3/users/25d0bce60985440cac97099ebd1a0f6c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:37.856422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d0bce60985440cac97099ebd1a0f6c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:37.858395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:38.028675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:38.112036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d0bce60985440cac97099ebd1a0f6c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:38.195102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d0bce60985440cac97099ebd1a0f6c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:38.213140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d0bce60985440cac97099ebd1a0f6c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:38.229973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d0bce60985440cac97099ebd1a0f6c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:38.231296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d0bce60985440cac97099ebd1a0f6c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:38.237372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 968/1921] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:37 2022] DELETE /identity/v3/users/25d0bce60985440cac97099ebd1a0f6c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:38.241979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:38.242496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:38.242593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:38.263615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:38.271174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:38.334548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 954/1922] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:38.345406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:38.346894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:38.420670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6d95d-3df4-4856-b4f6-6372bc1b8f3e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:38.425619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:38.426427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:38.427003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:38.427219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:38.438476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:38.440043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:38.453193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 969/1923] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:39.144346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:39.145832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:39.158594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10251ccd-7a96-4056-91c9-09429f4dfac0 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:39.164584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:39.165482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:39.165917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:39.166353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] PATH_INFO: `/v3/projects/7fa895f89a774e4caa29cfaf819b0666` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:39.167705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fa895f89a774e4caa29cfaf819b0666)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:39.169719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:39.228857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:39.305151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fa895f89a774e4caa29cfaf819b0666', 'request_id': 'req-0528004a-5462-4128-8126-b7eaf513fcdc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.313509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fa895f89a774e4caa29cfaf819b0666', 'request_id': 'req-0528004a-5462-4128-8126-b7eaf513fcdc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.405797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fa895f89a774e4caa29cfaf819b0666', 'request_id': 'req-0528004a-5462-4128-8126-b7eaf513fcdc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.418331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] The token cache is being invalidate because project 7fa895f89a774e4caa29cfaf819b0666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:39.418624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.419320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.421831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 955/1924] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:39 2022] DELETE /identity/v3/projects/7fa895f89a774e4caa29cfaf819b0666 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:39.425793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:39.438324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0a4cd-97e8-4b9b-b8dc-93198711e298 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:39.498579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:39.499234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:39.499507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:39.499781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] PATH_INFO: `/v3/users/92ea3ea2790a4e31a0c8485ad4090bc5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:39.500951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] RBAC: Authorizing `identity:delete_user(user_id=92ea3ea2790a4e31a0c8485ad4090bc5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:39.502973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:39.750325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92ea3ea2790a4e31a0c8485ad4090bc5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.879879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92ea3ea2790a4e31a0c8485ad4090bc5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.887723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92ea3ea2790a4e31a0c8485ad4090bc5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.900140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92ea3ea2790a4e31a0c8485ad4090bc5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.900857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-42a62c2a-e710-46fc-9b79-d1060df6235a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92ea3ea2790a4e31a0c8485ad4090bc5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:39.907676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 970/1925] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:39 2022] DELETE /identity/v3/users/92ea3ea2790a4e31a0c8485ad4090bc5 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:40.425857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:40.428042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:40.444590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528004a-5462-4128-8126-b7eaf513fcdc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:40.523335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:40.524032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:40.528169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:40.528681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] PATH_INFO: `/v3/projects/f068fc1a514a4be2bbb295fba3f8a2d9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:40.530094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f068fc1a514a4be2bbb295fba3f8a2d9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:40.532531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:40.674237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f068fc1a514a4be2bbb295fba3f8a2d9', 'request_id': 'req-d0cb8e1f-6075-4983-a712-05551f1fa7ee'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:40.680294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f068fc1a514a4be2bbb295fba3f8a2d9', 'request_id': 'req-d0cb8e1f-6075-4983-a712-05551f1fa7ee'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:40.703774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:40.788197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f068fc1a514a4be2bbb295fba3f8a2d9', 'request_id': 'req-d0cb8e1f-6075-4983-a712-05551f1fa7ee'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:40.800042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] The token cache is being invalidate because project f068fc1a514a4be2bbb295fba3f8a2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:40.800329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:40.800965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d0cb8e1f-6075-4983-a712-05551f1fa7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:40.802461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 956/1926] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:40 2022] DELETE /identity/v3/projects/f068fc1a514a4be2bbb295fba3f8a2d9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:40.807368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:40.807655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:40.807964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:40.829407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:40.830683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:40.895132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 971/1927] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:40.904106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:40.904452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:40.908897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:40.927333 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:40.928525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:40.936965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 957/1928] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:40.941252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:40.951538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ee5c-e0f2-4840-8a53-4ce303761db0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:40.956667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:40.957090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:40.957375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:40.957653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:40.958310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:40.958611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:40.960414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:40.973663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 972/1929] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:40.977829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:40.987737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8623f-b121-460b-a936-4c84ce778b24 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:40.992755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:40.994922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:40.994922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:40.994922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:40.994922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:40.996636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:41.096560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 958/1930] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:40 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:27:41.098196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:41.108930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338ba5f-ca9f-465a-b1c1-e3f6acc46746 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:41.165793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:41.166408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.166688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.166965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.167598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:41.169559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:41.327294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 973/1931] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:41 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:41.332267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:41.344707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861818b-360e-4b67-b52f-a60c4776556c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:41.349294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:41.349913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.350190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.350462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.351107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:41.352756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:41.364201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 959/1932] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:41.368345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:41.379677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d960-954c-4e67-9cf9-171d73314441 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:41.389005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0379cf4-51d6-4274-918f-d47049729753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:41.389676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0379cf4-51d6-4274-918f-d47049729753 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.389958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0379cf4-51d6-4274-918f-d47049729753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.390235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0379cf4-51d6-4274-918f-d47049729753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.391611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0379cf4-51d6-4274-918f-d47049729753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:41.393388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0379cf4-51d6-4274-918f-d47049729753 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:41.405409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 974/1933] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:41.409705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:41.420078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0619b-2bcf-4620-ae83-a356dac4cacf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:41.424473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:41.427557 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:41.428500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.428926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.429197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] PATH_INFO: `/v3/projects/c4476c447cc54249b067f8e4a0096ff7/users/d493f7b5606842faa748edc59eb07f35/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.447399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4476c447cc54249b067f8e4a0096ff7, user_id=d493f7b5606842faa748edc59eb07f35, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:41.450186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:41.566643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 960/1934] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:41 2022] PUT /identity/v3/projects/c4476c447cc54249b067f8e4a0096ff7/users/d493f7b5606842faa748edc59eb07f35/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:41.576930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.581593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.581593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.618797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] MFA Rules not processed for user `d493f7b5606842faa748edc59eb07f35`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:41.620105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:41.689588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 975/1935] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:41.694275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:41.766726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24860a75-0a6e-4611-af06-83f23c5ba7d6 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:41.772054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:41.772679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.772993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.773270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.773923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:41.775916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:41.905229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 961/1936] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:41 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:41.909448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:41.920245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da2e35-8c9a-49e3-99fd-4a09e0b18a6d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:41.982648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:41.984649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:41.984649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:41.984649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:41.985091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:41.987074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:42.143711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 976/1937] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:41 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:42.148302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:42.159234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee018ed-f9fd-4444-82c2-463c5609fd1b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:42.164214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:42.164944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:42.165051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:42.165264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:42.165901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:42.167497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:42.180864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 962/1938] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:42.185032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:42.195398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3036e3c-3a81-4ee3-8b29-3006205b22b8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:42.200260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157a75d-ffea-46a9-8b93-29a320c13fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:42.200908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157a75d-ffea-46a9-8b93-29a320c13fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:42.201210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157a75d-ffea-46a9-8b93-29a320c13fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:42.201485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157a75d-ffea-46a9-8b93-29a320c13fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:42.202183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157a75d-ffea-46a9-8b93-29a320c13fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:42.203904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f157a75d-ffea-46a9-8b93-29a320c13fb8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:42.215595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 977/1939] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:42.220549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:42.230345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983a9f0-c403-45a9-9444-64356091c0a1 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:42.236192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:42.236877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:42.238073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:42.238073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] PATH_INFO: `/v3/projects/b2a0266889184410ac1a297171d16573/users/57e116563d8d435fa3e515865afc01ae/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:42.252543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a0266889184410ac1a297171d16573, user_id=57e116563d8d435fa3e515865afc01ae, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:42.255478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:42.377658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 963/1940] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:42 2022] PUT /identity/v3/projects/b2a0266889184410ac1a297171d16573/users/57e116563d8d435fa3e515865afc01ae/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:42.387969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575625-6418-43b7-800f-4944f6602bef None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:42.388260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575625-6418-43b7-800f-4944f6602bef None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:42.388543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a575625-6418-43b7-800f-4944f6602bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:42.419490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8a575625-6418-43b7-800f-4944f6602bef None None] MFA Rules not processed for user `57e116563d8d435fa3e515865afc01ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:42.420658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8a575625-6418-43b7-800f-4944f6602bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:42.487766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 978/1941] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:42.492278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:42.558449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590a024-33fc-4097-9ccd-b016d27f5fa9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:42.563280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:42.563929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:42.564280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:42.564578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:42.565259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:42.567242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:42.716519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 964/1942] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:42 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:42.724910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575625-6418-43b7-800f-4944f6602bef None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:42.736967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a575625-6418-43b7-800f-4944f6602bef None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:42.797979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:42.798504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:42.798786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:42.799178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:42.799816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:42.802086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.007412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 979/1943] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:42 2022] POST /identity/v3/users => generated 295 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:43.011356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:43.022649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a9888-5509-4821-b5e2-bdad2b48c34a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:43.027282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:43.027908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.028198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.028475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.029116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:43.030733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.042649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 965/1944] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:27:43.046927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:43.057808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e058a1-1c83-4c7d-90dd-68c12d13a02f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:43.063120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:43.063789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.064104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.064381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] PATH_INFO: `/v3/projects/a16136c526cc4825ac336c88b2931b3d/users/7d4e2d80546849d0b25bb182553e3d9d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.081431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16136c526cc4825ac336c88b2931b3d, user_id=7d4e2d80546849d0b25bb182553e3d9d, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:43.084211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.228945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 980/1945] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:43 2022] PUT /identity/v3/projects/a16136c526cc4825ac336c88b2931b3d/users/7d4e2d80546849d0b25bb182553e3d9d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:43.232853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:43.243598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf231e-ccc8-419c-87a1-4ec6c1abfd2c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:43.300483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd6fa6-05f9-462b-b09e-ed127e8163f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:43.301502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd6fa6-05f9-462b-b09e-ed127e8163f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.301955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd6fa6-05f9-462b-b09e-ed127e8163f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.302392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd6fa6-05f9-462b-b09e-ed127e8163f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.303228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd6fa6-05f9-462b-b09e-ed127e8163f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:43.305141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd6fa6-05f9-462b-b09e-ed127e8163f6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.319275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 966/1946] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:43.323706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:43.335452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572913e3-e12a-4bb8-b86f-6b2135d831c9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:43.340127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:43.340833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.341117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.341397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] PATH_INFO: `/v3/projects/a16136c526cc4825ac336c88b2931b3d/users/7d4e2d80546849d0b25bb182553e3d9d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.343421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16136c526cc4825ac336c88b2931b3d, user_id=7d4e2d80546849d0b25bb182553e3d9d, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:43.346085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.356967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 981/1947] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:43 2022] PUT /identity/v3/projects/a16136c526cc4825ac336c88b2931b3d/users/7d4e2d80546849d0b25bb182553e3d9d/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:43.368972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.369260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.369539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.398254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] MFA Rules not processed for user `7d4e2d80546849d0b25bb182553e3d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:43.399018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:43.473872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 967/1948] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:43.486233 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:43.487986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:43.554604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f39dc7-6ed4-4da5-8003-47af8a6c70f5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:43.558333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:43.558942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.559289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.559551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.569764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:43.571216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.580956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 982/1949] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:43.701000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:43.702522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:43.714435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340880b-4b98-4648-a593-0e2cb2185de4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:43.718102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:43.719213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:43.719494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:43.719767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:43.776495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:43.777882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:43.787830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 968/1950] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:44.081434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:44.828417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:44.869468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:44.878597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5461bf-677d-41c9-a719-31d102dcaa9a service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:44.882313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:44.882938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:44.883236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:44.883511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:44.885295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:44.959518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:44.961323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5724543-6228-4420-8847-4782c785d4b2 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:44.972308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 983/1951] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:45.994310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:46.228758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:47.171441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:47.354102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:48.017008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:48.474503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:49.663443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:49.800359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:52.053825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:52.125309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9b4b5-869c-4b9b-90be-860079b2b96c service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:52.130520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:52.131178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:52.131462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:52.131739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] PATH_INFO: `/v3/users/d493f7b5606842faa748edc59eb07f35` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:52.133160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d493f7b5606842faa748edc59eb07f35)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:52.134953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:52.205928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:52.418683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd493f7b5606842faa748edc59eb07f35'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:52.419719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd493f7b5606842faa748edc59eb07f35'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:52.433990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd493f7b5606842faa748edc59eb07f35'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:52.434721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd493f7b5606842faa748edc59eb07f35'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:52.442570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd493f7b5606842faa748edc59eb07f35'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:52.477460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:52.596235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 969/1952] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:52 2022] DELETE /identity/v3/users/d493f7b5606842faa748edc59eb07f35 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:52.600864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:52.601183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:52.601461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:52.620614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:52.621834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:52.681708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 984/1953] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:52.691931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:52.693363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:52.748792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198b0ab-2e41-415f-a102-c8b03bf9498b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:52.753174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:52.753821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:52.754099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:52.754377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:52.763599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:52.765184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:52.775642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 970/1954] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:53.316366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:53.318055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:53.340887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d1e39-f826-4fcc-ae41-4a01865a09c9 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:53.348915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:53.353620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:53.353620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:53.353620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] PATH_INFO: `/v3/projects/c4476c447cc54249b067f8e4a0096ff7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:53.353620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4476c447cc54249b067f8e4a0096ff7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:53.353620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:53.472293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4476c447cc54249b067f8e4a0096ff7', 'request_id': 'req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.479800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4476c447cc54249b067f8e4a0096ff7', 'request_id': 'req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.571715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4476c447cc54249b067f8e4a0096ff7', 'request_id': 'req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.584940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] The token cache is being invalidate because project c4476c447cc54249b067f8e4a0096ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:53.585423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.586137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.588175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 985/1955] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:53 2022] DELETE /identity/v3/projects/c4476c447cc54249b067f8e4a0096ff7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:53.592465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:53.604096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94054c0c-128f-45d0-8848-995e068986d5 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:53.661213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:53.661845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:53.662123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:53.662394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] PATH_INFO: `/v3/users/57e116563d8d435fa3e515865afc01ae` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:53.663508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e116563d8d435fa3e515865afc01ae)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:53.665473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:53.704512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:53.848175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e116563d8d435fa3e515865afc01ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.848855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e116563d8d435fa3e515865afc01ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.862683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e116563d8d435fa3e515865afc01ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.863399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e116563d8d435fa3e515865afc01ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.870617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e116563d8d435fa3e515865afc01ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:53.989434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 971/1956] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:53 2022] DELETE /identity/v3/users/57e116563d8d435fa3e515865afc01ae => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:54.326015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:54.441246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:54.442659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:54.463432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3dba1-52f0-43fd-a459-161e7ed9ec84 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:54.524541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:54.525281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:54.525478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:54.525762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] PATH_INFO: `/v3/projects/b2a0266889184410ac1a297171d16573` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:54.528071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a0266889184410ac1a297171d16573)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:54.529667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:54.650309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:54.655013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a0266889184410ac1a297171d16573', 'request_id': 'req-cce6f182-306d-4955-97ce-f77c596ffe8b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:54.662777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a0266889184410ac1a297171d16573', 'request_id': 'req-cce6f182-306d-4955-97ce-f77c596ffe8b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:54.805024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a0266889184410ac1a297171d16573', 'request_id': 'req-cce6f182-306d-4955-97ce-f77c596ffe8b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:54.817515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] The token cache is being invalidate because project b2a0266889184410ac1a297171d16573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:54.817808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:54.818444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:54.820153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 986/1957] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:54 2022] DELETE /identity/v3/projects/b2a0266889184410ac1a297171d16573 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:54.824278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:54.825734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:54.840808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b174f-8eb1-4fd4-bf99-e0c20e339e21 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:54.906498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:54.907185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:54.907431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:54.907709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] PATH_INFO: `/v3/users/7d4e2d80546849d0b25bb182553e3d9d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:54.909584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4e2d80546849d0b25bb182553e3d9d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:54.911525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:55.119009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4e2d80546849d0b25bb182553e3d9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:55.119712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4e2d80546849d0b25bb182553e3d9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:55.134576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4e2d80546849d0b25bb182553e3d9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:55.135584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4e2d80546849d0b25bb182553e3d9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:55.145252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4e005403-68bd-449f-b059-a83e193362ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4e2d80546849d0b25bb182553e3d9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:55.241054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 972/1958] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:54 2022] DELETE /identity/v3/users/7d4e2d80546849d0b25bb182553e3d9d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:55.790102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:55.791523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:55.805379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6f182-306d-4955-97ce-f77c596ffe8b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:55.866185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:55.866860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:55.867145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:55.867426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] PATH_INFO: `/v3/projects/a16136c526cc4825ac336c88b2931b3d` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:55.868686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16136c526cc4825ac336c88b2931b3d)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:55.870689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:55.943091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:55.994841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16136c526cc4825ac336c88b2931b3d', 'request_id': 'req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:56.000817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16136c526cc4825ac336c88b2931b3d', 'request_id': 'req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:56.074252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16136c526cc4825ac336c88b2931b3d', 'request_id': 'req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:56.087067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] The token cache is being invalidate because project a16136c526cc4825ac336c88b2931b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:56.087353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:56.088045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4334a5a6-d1b6-4ddc-b279-acdfa0438b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:56.089539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 987/1959] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:55 2022] DELETE /identity/v3/projects/a16136c526cc4825ac336c88b2931b3d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:56.095685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.096193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.096282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.116633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:56.117842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:56.178389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 973/1960] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:56.186715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.187004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.188091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.210454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:56.212006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:56.220336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 988/1961] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:56.225493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.236732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeefd2-b042-4024-8f2d-6ba14519f983 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.241223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.242797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.242797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.242797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.243096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:56.243405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.245111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.258279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 974/1962] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:56.262841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.268527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:56.272310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e318f0-b180-4b54-aa2b-255f8cf88142 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.277209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.277829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.278105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.278376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.279022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.281098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.418802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 989/1963] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:56 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:56.423732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.434064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d958c-5b44-4409-b15b-614848484f03 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.491722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.492371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.492653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.492929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.493570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.495564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.640211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 975/1964] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:56 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:56.644553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.657283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03673e2f-560e-4069-b6a0-a8156859c774 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.662055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.662692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.662993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.663258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.663968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.665615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.685634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 990/1965] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:56.691200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.702471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d7564-df1e-40fb-b5f0-486f8f3820bf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.707332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc9cb1-bb45-442e-b8e8-340f8fee88a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.708020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc9cb1-bb45-442e-b8e8-340f8fee88a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.708320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc9cb1-bb45-442e-b8e8-340f8fee88a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.708654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc9cb1-bb45-442e-b8e8-340f8fee88a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.709294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc9cb1-bb45-442e-b8e8-340f8fee88a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.710917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc9cb1-bb45-442e-b8e8-340f8fee88a3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.724378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 976/1966] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:56.731776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:56.751766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0caf2b-6d15-4ce1-bac3-1269284ae74e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:56.756147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:56.756824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.757136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.757413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] PATH_INFO: `/v3/projects/078875891e734628b2039371b6e2abcc/users/0f2a6c5ab8fa4dc89d5c77a79d5234b5/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.775270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=078875891e734628b2039371b6e2abcc, user_id=0f2a6c5ab8fa4dc89d5c77a79d5234b5, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:56.778214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:56.898519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 991/1967] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:56 2022] PUT /identity/v3/projects/078875891e734628b2039371b6e2abcc/users/0f2a6c5ab8fa4dc89d5c77a79d5234b5/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:56.911721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:56.912133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:56.912425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:56.941026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] MFA Rules not processed for user `0f2a6c5ab8fa4dc89d5c77a79d5234b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:56.942275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:57.012155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 977/1968] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:57.024705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:57.026512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:57.089360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c22fe-40bc-4b39-927f-ee339aca58cf admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:57.093861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548fe0c-0874-4690-a6f1-814311bda19d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:57.094497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548fe0c-0874-4690-a6f1-814311bda19d service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.094784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548fe0c-0874-4690-a6f1-814311bda19d service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.095620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548fe0c-0874-4690-a6f1-814311bda19d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.106007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548fe0c-0874-4690-a6f1-814311bda19d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:57.107595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548fe0c-0874-4690-a6f1-814311bda19d service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:57.119626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 992/1969] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:27:57.144392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:57.214732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4268-39c6-4a95-92d6-826fe8937c46 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:57.219435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:57.220171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.220453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.220733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] PATH_INFO: `/v3/users/0f2a6c5ab8fa4dc89d5c77a79d5234b5` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.222447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2a6c5ab8fa4dc89d5c77a79d5234b5)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:57.223865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:57.436786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2a6c5ab8fa4dc89d5c77a79d5234b5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:57.438074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2a6c5ab8fa4dc89d5c77a79d5234b5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:57.450235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2a6c5ab8fa4dc89d5c77a79d5234b5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:57.450845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2a6c5ab8fa4dc89d5c77a79d5234b5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:57.459182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2a6c5ab8fa4dc89d5c77a79d5234b5'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:57.577468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:57.606316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 978/1970] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:57 2022] DELETE /identity/v3/users/0f2a6c5ab8fa4dc89d5c77a79d5234b5 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:57.616085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.616382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.616665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.638321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:57.639754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:57.718913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 993/1971] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:57.730998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:57.732502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:57.804137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6834ff7-6669-4c62-b595-dda3ef2391a8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:57.808482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca1ddc-d262-4b8b-8c51-9402f019819b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:57.809284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca1ddc-d262-4b8b-8c51-9402f019819b service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:57.809569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca1ddc-d262-4b8b-8c51-9402f019819b service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:57.809902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca1ddc-d262-4b8b-8c51-9402f019819b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:57.821718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca1ddc-d262-4b8b-8c51-9402f019819b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:57.823991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca1ddc-d262-4b8b-8c51-9402f019819b service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:57.836079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 979/1972] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:58.045979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:58.401763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:58.404779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:58.426100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7942b-2af6-45f8-8c4b-715a89d805a5 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:58.431214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:58.431864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.432177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.432466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] PATH_INFO: `/v3/projects/078875891e734628b2039371b6e2abcc` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.433583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=078875891e734628b2039371b6e2abcc)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:58.435540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:58.572560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078875891e734628b2039371b6e2abcc', 'request_id': 'req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:58.579092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078875891e734628b2039371b6e2abcc', 'request_id': 'req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:58.674323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078875891e734628b2039371b6e2abcc', 'request_id': 'req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:58.687991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] The token cache is being invalidate because project 078875891e734628b2039371b6e2abcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:27:58.688287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:58.688953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-170409b4-0caf-43a0-bfa2-6e3d4b97a85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:58.691001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 994/1973] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:27:58 2022] DELETE /identity/v3/projects/078875891e734628b2039371b6e2abcc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:58.695820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.696348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.696440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.717264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:58.733365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:58.777696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 980/1974] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:58.785500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.785794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.786072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.806183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:58.807439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:58.817433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 995/1975] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:58.821730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:58.832502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640a0a2-e219-4f32-bcd0-69e39b7b08b2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:58.837673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:58.838293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.838708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.838973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.839638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:27:58.839986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:58.841761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:58.854496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 981/1976] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:27:58.859237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:58.868814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e723d5-b9d8-4562-91c7-6f1bd74dca6c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:58.873720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:58.874401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:58.875124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:58.875403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:58.876084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:58.878013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:58.980001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 996/1977] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:27:58 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:58.983748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:58.994567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187827e-bffe-4f06-b36e-679210794fe8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.054342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.054960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.055262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.055501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.056225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.057699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:27:59.058365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.160938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 982/1978] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:27:58 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:27:59.164991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.177339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217adb7-297d-4844-add3-7c91f97dee49 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.182910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.183462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.183744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.184094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.184839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.186604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.201099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 997/1979] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:59.212028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.214645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.216452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.227078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b726eac-9235-40c9-a601-71f13752eb56 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.229837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452c397-2cc4-4ee3-92b9-12c73dea9ea3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.231851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.232496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.232890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.233068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.233677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.235670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.235844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.236469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.236874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.237272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] PATH_INFO: `/v3/users/b897d61b7aae4a62998a6cc469880a24` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.238544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b897d61b7aae4a62998a6cc469880a24)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.240625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.248965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 983/1980] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:27:59.253706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.263521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e45cf-d68f-46e7-a5f2-ef30bd702c65 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.268411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436934f7-44b3-4344-8afc-02e032407ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.269062 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436934f7-44b3-4344-8afc-02e032407ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.269344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436934f7-44b3-4344-8afc-02e032407ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.269619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436934f7-44b3-4344-8afc-02e032407ba2 admin admin] PATH_INFO: `/v3/projects/5f0382c560dd4a8cbd4cacc29e97823d/users/fda00e4a39a1494ba8c083d51e106934/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.287659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436934f7-44b3-4344-8afc-02e032407ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0382c560dd4a8cbd4cacc29e97823d, user_id=fda00e4a39a1494ba8c083d51e106934, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.289499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436934f7-44b3-4344-8afc-02e032407ba2 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.476068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 984/1981] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:27:59 2022] PUT /identity/v3/projects/5f0382c560dd4a8cbd4cacc29e97823d/users/fda00e4a39a1494ba8c083d51e106934/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:27:59.486935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.487274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.487554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.523211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] MFA Rules not processed for user `fda00e4a39a1494ba8c083d51e106934`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:59.524536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.603966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 985/1982] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:27:59.617734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b897d61b7aae4a62998a6cc469880a24'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:59.641581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.649403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.708191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b897d61b7aae4a62998a6cc469880a24'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:59.712711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22797567-90d2-4ef4-b873-7299aa854256 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.716610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.717167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.726155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.726155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.726371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b897d61b7aae4a62998a6cc469880a24'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:59.732484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.733494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b897d61b7aae4a62998a6cc469880a24'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:59.733967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.734814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4ba3c410-1ac6-4566-8f30-e69603d959b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b897d61b7aae4a62998a6cc469880a24'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:27:59.742410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 998/1983] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:27:59 2022] DELETE /identity/v3/users/b897d61b7aae4a62998a6cc469880a24 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:27:59.746997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.747143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 986/1984] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:27:59.747589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.747879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.769503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:27:59.770743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.837775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 999/1985] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:27:59.848883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:27:59.850850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:27:59.918489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687500ad-82ea-4a54-9405-95a504ad63e4 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:27:59.974374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:27:59.982496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:27:59.982496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:27:59.982496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:27:59.984953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:27:59.986417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:27:59.996510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 987/1986] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:00.571854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:00.585263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:00.587306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:00.618083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca653a3-d452-4cfa-b827-10586cdba17d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:00.640811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:00.642372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:00.642798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:00.643152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] PATH_INFO: `/v3/projects/c3a396cb6ec74c9c9df40b29d4b84c1e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:00.644580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a396cb6ec74c9c9df40b29d4b84c1e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:00.647341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:00.833965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:00.834934 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a396cb6ec74c9c9df40b29d4b84c1e', 'request_id': 'req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:00.841660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a396cb6ec74c9c9df40b29d4b84c1e', 'request_id': 'req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.034760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a396cb6ec74c9c9df40b29d4b84c1e', 'request_id': 'req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.051363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] The token cache is being invalidate because project c3a396cb6ec74c9c9df40b29d4b84c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:01.052670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.056448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.058750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1000/1987] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:00 2022] DELETE /identity/v3/projects/c3a396cb6ec74c9c9df40b29d4b84c1e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:01.063012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:01.079227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b60aa0-0b3f-48b3-bece-918e0ef8fbdf service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:01.145368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:01.145965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:01.146246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:01.146522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] PATH_INFO: `/v3/users/9d192bf2abb449c1b1c0e89b8827f928` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:01.147670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d192bf2abb449c1b1c0e89b8827f928)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:01.149721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:01.402863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d192bf2abb449c1b1c0e89b8827f928'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.404091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d192bf2abb449c1b1c0e89b8827f928'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.419533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d192bf2abb449c1b1c0e89b8827f928'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.420260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d192bf2abb449c1b1c0e89b8827f928'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.427613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fd48ed8a-9e6b-41ec-a4f0-70830392b5be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d192bf2abb449c1b1c0e89b8827f928'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:01.572732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 988/1988] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:01 2022] DELETE /identity/v3/users/9d192bf2abb449c1b1c0e89b8827f928 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:02.198860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:02.198860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:02.214574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea2d4b-3d82-42e7-926d-f7c2d6c56445 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:02.301510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:02.301510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.301510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.301510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] PATH_INFO: `/v3/projects/e645c9435a0e41f1a066e9edeee6ff5c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.302046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e645c9435a0e41f1a066e9edeee6ff5c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:02.302046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:02.460018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e645c9435a0e41f1a066e9edeee6ff5c', 'request_id': 'req-6adae136-aeb2-483a-a6b5-739960bebd84'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:02.466046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e645c9435a0e41f1a066e9edeee6ff5c', 'request_id': 'req-6adae136-aeb2-483a-a6b5-739960bebd84'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:02.542898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e645c9435a0e41f1a066e9edeee6ff5c', 'request_id': 'req-6adae136-aeb2-483a-a6b5-739960bebd84'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:02.555621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] The token cache is being invalidate because project e645c9435a0e41f1a066e9edeee6ff5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:02.556008 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:02.557291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6adae136-aeb2-483a-a6b5-739960bebd84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:02.558439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1001/1989] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:02 2022] DELETE /identity/v3/projects/e645c9435a0e41f1a066e9edeee6ff5c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:02.565029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.565561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.565561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.586182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:02.587646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:02.656331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 989/1990] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:02.656331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:02.658355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.665378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.665744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.687852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:02.689111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:02.698166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1002/1991] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:02.702347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:02.713138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df57a3-9975-491c-9402-38ab48bfd0f7 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:02.717721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:02.718409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.718690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.719176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.719828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:02.720160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:02.720273 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:02.722078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:02.737774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 990/1992] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:02.742588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:02.753879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48faa0ed-6e9e-4be5-8b91-e6ef1d8f42d8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:02.758862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:02.759649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.760101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.760599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.761681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:02.763872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:02.845914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1003/1993] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:02 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:02.850475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:02.868803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d75729-7c29-428d-9d52-85297667f3d9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:02.944775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:02.952910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:02.952910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:02.952910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:02.952910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:02.952910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.111599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 991/1994] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:02 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:03.115388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.130169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55cd77-0e29-41b3-87f9-126157cffb43 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.144971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.145611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.145895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.153766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.153766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.153766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.167161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1004/1995] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:03.168412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.179987 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049582a-d16d-4f94-b626-91d66496abf6 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.185107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4365dd-4545-4584-b0d5-8d49d0cf5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.185728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4365dd-4545-4584-b0d5-8d49d0cf5814 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.186069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4365dd-4545-4584-b0d5-8d49d0cf5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.186350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4365dd-4545-4584-b0d5-8d49d0cf5814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.187102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4365dd-4545-4584-b0d5-8d49d0cf5814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.193537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4365dd-4545-4584-b0d5-8d49d0cf5814 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.234771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 992/1996] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:03.237810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.252815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165091d-fef3-4268-8c8a-b24dfc539b5b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.258105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.258713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.258997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.259304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] PATH_INFO: `/v3/projects/f7bcfe31b87d44ef9e0106ecd4d68bad/users/b5b547711d184917a2d5616d95fdf3b8/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.279951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bcfe31b87d44ef9e0106ecd4d68bad, user_id=b5b547711d184917a2d5616d95fdf3b8, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.282867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.366871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1005/1997] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:03 2022] PUT /identity/v3/projects/f7bcfe31b87d44ef9e0106ecd4d68bad/users/b5b547711d184917a2d5616d95fdf3b8/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:03.380831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.381251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.381609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.416866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] MFA Rules not processed for user `b5b547711d184917a2d5616d95fdf3b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:03.418162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:03.490699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 993/1998] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:03.495464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.558003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71b9d4-9e0f-4dd7-a1ea-02fb1028f58e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.563342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.564014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.564318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.564595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.565237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.567209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.664370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1006/1999] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:03 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:03.668493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.680729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73da62-13e2-499c-a6b9-d41b485af65f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.745239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.746503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.746952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.747385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.748343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.751367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.852143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:03.887471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 994/2000] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:03 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:03.893343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.904811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c8f74-0422-4efe-ac6f-483bddc2d147 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.910966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.911586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.911871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.912243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.912925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.914590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.926243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1007/2001] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:03.930297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.941085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7687eb-518c-40c3-a019-6b94728f57a7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.945392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507cf4-6545-49a8-89b4-20db5e0adc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.946003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507cf4-6545-49a8-89b4-20db5e0adc2b admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.946279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507cf4-6545-49a8-89b4-20db5e0adc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.946561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507cf4-6545-49a8-89b4-20db5e0adc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:03.947193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507cf4-6545-49a8-89b4-20db5e0adc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:03.948834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507cf4-6545-49a8-89b4-20db5e0adc2b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:03.960829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 995/2002] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:03.967393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:03.980085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221286e3-4074-4027-ba4d-ededec25f6f4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:03.985684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:03.986375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:03.986662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:03.987317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] PATH_INFO: `/v3/projects/39721950a9ca4e32a9016ed501e68da2/users/66a56512959e495eab4686e66a47eaec/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.004052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39721950a9ca4e32a9016ed501e68da2, user_id=66a56512959e495eab4686e66a47eaec, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:04.018781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:04.092123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1008/2003] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:03 2022] PUT /identity/v3/projects/39721950a9ca4e32a9016ed501e68da2/users/66a56512959e495eab4686e66a47eaec/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:04.103013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:04.103550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:04.103968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.149256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] MFA Rules not processed for user `66a56512959e495eab4686e66a47eaec`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:04.150481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:04.166057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:04.224040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 996/2004] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:04.225964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:04.323701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e417a-fee2-4a2e-be89-f91d0b5ef745 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:04.330039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:04.330661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:04.331118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:04.331392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.332101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:04.334065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:04.445370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1009/2005] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:04 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:04.459540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:04.496862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5494c27-a509-4439-8b19-887e196b1985 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:04.653949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:04.655207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:04.655590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:04.656236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.657673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:04.661504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:04.770388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 997/2006] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:04 2022] POST /identity/v3/users => generated 295 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:04.770993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:04.783605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712d8b4-0e04-4f0b-8f9c-233ba8829619 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:04.790501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:04.792062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:04.792505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:04.792966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.794095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:04.796561 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:04.816316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1010/2007] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:04.817429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:04.829957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322871ab-2b98-464c-801e-f3d1d7f7c25a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:04.832241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:04.838144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:04.839068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:04.839427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:04.839957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] PATH_INFO: `/v3/projects/37cfb839e44c46a385d08c8b2677e7fc/users/1d94b485940b498dab092551742d97d3/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:04.857893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cfb839e44c46a385d08c8b2677e7fc, user_id=1d94b485940b498dab092551742d97d3, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:04.860890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:04.969764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 998/2008] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:04 2022] PUT /identity/v3/projects/37cfb839e44c46a385d08c8b2677e7fc/users/1d94b485940b498dab092551742d97d3/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:04.971958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:04.987352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b774a7d-eb3b-4638-a0ac-9ff13bce2bd9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:04.990661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:05.058747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccc113-7254-4a5e-9cd2-a150a709a3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:05.059494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccc113-7254-4a5e-9cd2-a150a709a3ec admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.059846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccc113-7254-4a5e-9cd2-a150a709a3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.060152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccc113-7254-4a5e-9cd2-a150a709a3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.060795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccc113-7254-4a5e-9cd2-a150a709a3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:05.063280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccc113-7254-4a5e-9cd2-a150a709a3ec admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:05.077002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1011/2009] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:05.078353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:05.089499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85965f4-1a50-43dd-ab36-23cdc170352a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:05.097090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:05.097836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.098197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.098558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] PATH_INFO: `/v3/projects/37cfb839e44c46a385d08c8b2677e7fc/users/1d94b485940b498dab092551742d97d3/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.100795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cfb839e44c46a385d08c8b2677e7fc, user_id=1d94b485940b498dab092551742d97d3, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:05.103545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:05.115464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 999/2010] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:05 2022] PUT /identity/v3/projects/37cfb839e44c46a385d08c8b2677e7fc/users/1d94b485940b498dab092551742d97d3/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:05.127603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22346b2-0f4c-4f45-8d59-93f831d8073b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.127913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22346b2-0f4c-4f45-8d59-93f831d8073b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.128215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22346b2-0f4c-4f45-8d59-93f831d8073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.156676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e22346b2-0f4c-4f45-8d59-93f831d8073b None None] MFA Rules not processed for user `1d94b485940b498dab092551742d97d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:05.157896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e22346b2-0f4c-4f45-8d59-93f831d8073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.234791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1012/2011] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:05.246921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:05.248836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.302619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaac60-d406-44f3-b527-79e9f5e708bb None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.303012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaac60-d406-44f3-b527-79e9f5e708bb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.303370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaac60-d406-44f3-b527-79e9f5e708bb None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.305202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1013/2012] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:28:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:28:05.315394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f78d1-5de0-4a34-bfe3-3e8687704176 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:05.320859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60db1a-e917-411b-bbe0-71be1da4db6f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.321988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60db1a-e917-411b-bbe0-71be1da4db6f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.322689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5711-2513-4db1-a2dc-fc2493ea5849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:05.322809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60db1a-e917-411b-bbe0-71be1da4db6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.323175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5711-2513-4db1-a2dc-fc2493ea5849 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.323491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5711-2513-4db1-a2dc-fc2493ea5849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.323803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5711-2513-4db1-a2dc-fc2493ea5849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.354912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-dd60db1a-e917-411b-bbe0-71be1da4db6f None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:05.356434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-dd60db1a-e917-411b-bbe0-71be1da4db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.413706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c5711-2513-4db1-a2dc-fc2493ea5849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:05.415161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c5711-2513-4db1-a2dc-fc2493ea5849 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:05.424747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1000/2013] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:05.429147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1014/2014] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:05.441281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b4b80-a98f-4000-a2f6-37a8a4cd7c70 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.441573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b4b80-a98f-4000-a2f6-37a8a4cd7c70 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.441970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b4b80-a98f-4000-a2f6-37a8a4cd7c70 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.443375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1001/2015] 10.222.0.65 () {54 vars in 961 bytes} [Mon Aug 29 21:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:28:05.448544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.448837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.449112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.470887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b4b80-a98f-4000-a2f6-37a8a4cd7c70 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:05.473006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bc0b4b80-a98f-4000-a2f6-37a8a4cd7c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.488966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:05.491130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:05.503664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1015/2016] 10.222.0.65 () {58 vars in 1058 bytes} [Mon Aug 29 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:05.512891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:05.524804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26b686-7bb5-4a1b-88c5-237b72e6283c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:05.530490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:05.531996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.532382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.532735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.541835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:05.543528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:05.555548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b4b80-a98f-4000-a2f6-37a8a4cd7c70 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:05.555727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1016/2017] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:05.565498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:05.565871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:05.566151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:05.566426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] PATH_INFO: `/v3/users/e135d9b5894147e6a391d75d8896f094` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:05.567601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e135d9b5894147e6a391d75d8896f094)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:05.569766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:05.837302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e135d9b5894147e6a391d75d8896f094'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:05.838587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e135d9b5894147e6a391d75d8896f094'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:05.859486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e135d9b5894147e6a391d75d8896f094'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:05.859486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e135d9b5894147e6a391d75d8896f094'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:05.865335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7ed43534-825d-4a51-9176-9bea8715f713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e135d9b5894147e6a391d75d8896f094'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:05.960572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1002/2018] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:05 2022] DELETE /identity/v3/users/e135d9b5894147e6a391d75d8896f094 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:06.293841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:06.304467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bd8e8-e9bd-4e98-8a2a-b08ac2591c36 service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:06.363205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:06.363872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.364061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.364338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] PATH_INFO: `/v3/projects/c852db97ba734c7088df6af689b41d16` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.365563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c852db97ba734c7088df6af689b41d16)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:06.367474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:06.434145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:06.475751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c852db97ba734c7088df6af689b41d16', 'request_id': 'req-4066c432-b90b-43a9-8f44-8a094d0ecaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:06.481473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c852db97ba734c7088df6af689b41d16', 'request_id': 'req-4066c432-b90b-43a9-8f44-8a094d0ecaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:06.577073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c852db97ba734c7088df6af689b41d16', 'request_id': 'req-4066c432-b90b-43a9-8f44-8a094d0ecaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:06.591039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] The token cache is being invalidate because project c852db97ba734c7088df6af689b41d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:06.591332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:06.592055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4066c432-b90b-43a9-8f44-8a094d0ecaec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:06.593770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1017/2019] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:06 2022] DELETE /identity/v3/projects/c852db97ba734c7088df6af689b41d16 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:06.600270 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.600564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.600846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.614098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:06.623817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:06.625657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:06.689529 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1003/2020] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:06.702093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.702474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.702845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.723831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:06.725576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:06.734777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1018/2021] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:06.738728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:06.748594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f82c8d-bde9-44d2-acc9-0d2243dbd287 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:06.754290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:06.754869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.755154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.755431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.756295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:06.756600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:06.763983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:06.773702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1004/2022] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:06.778011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:06.787656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72650d9-87f3-43a2-b446-bd90083abb75 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:06.792407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:06.793012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:06.793296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:06.793575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:06.796965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:06.796965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:06.969818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1019/2023] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:06 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:06.978204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:06.990245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc913e-8f1e-480f-9cc0-0b7409014247 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.081462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.082273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.082553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.082826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.083523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.085508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.178020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1005/2024] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:06 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:07.182033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.192470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f88486-37bc-40de-86c2-2c9a4abc29d8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.197261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.197872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.198157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.198433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.199078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.200730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.214543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1020/2025] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:07.215731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.229814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff8ff4-46e7-4118-b656-6e3edaa8edae admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.235122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0975c-6b44-46f0-9e30-8d2b01b08fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.236058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0975c-6b44-46f0-9e30-8d2b01b08fda admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.236341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0975c-6b44-46f0-9e30-8d2b01b08fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.236615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0975c-6b44-46f0-9e30-8d2b01b08fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.237265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0975c-6b44-46f0-9e30-8d2b01b08fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.243464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0975c-6b44-46f0-9e30-8d2b01b08fda admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.251221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1006/2026] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:07.255629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.266997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc5f89-bbcb-4aa7-bb61-c696cfb99e4d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.273884 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.274639 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.275002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.275282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] PATH_INFO: `/v3/projects/5bc75af3ba6c442889ce29351414e110/users/0e3f8c861a9c49119fc531e2a159ec9d/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.291845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc75af3ba6c442889ce29351414e110, user_id=0e3f8c861a9c49119fc531e2a159ec9d, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.294579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.384568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1021/2027] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:07 2022] PUT /identity/v3/projects/5bc75af3ba6c442889ce29351414e110/users/0e3f8c861a9c49119fc531e2a159ec9d/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:07.389081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.389378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.389657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.414856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:07.416539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.483412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1007/2028] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:07.534584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.536529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.538859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.540285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.594663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c2aca-0d35-4711-bff1-eba929bc4735 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.599046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5d609-1e68-4722-ae48-f1dfb7e121c0 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.599236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.599686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.600196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.600479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.604037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.604747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.605032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.605324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.607676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.611157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.612656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.622647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1022/2029] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:07.673213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.674554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672d14f-e53b-4690-a803-b752caf4421f service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.683938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1008/2030] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:07.731615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:07.732734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:07.745464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70d711-651d-483d-91f7-75109275b0f4 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:07.750868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:07.751516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:07.751799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:07.752119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] PATH_INFO: `/v3/users/f1ee0b947bd840ddb838ad243ede51ed` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:07.753282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ee0b947bd840ddb838ad243ede51ed)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:07.755278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:07.979333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ee0b947bd840ddb838ad243ede51ed'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:08.060374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ee0b947bd840ddb838ad243ede51ed'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:08.070053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ee0b947bd840ddb838ad243ede51ed'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:08.083291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ee0b947bd840ddb838ad243ede51ed'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:08.084122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ee0b947bd840ddb838ad243ede51ed'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:08.091424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1023/2031] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:07 2022] DELETE /identity/v3/users/f1ee0b947bd840ddb838ad243ede51ed => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:08.096578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:08.097093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:08.097582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:08.119908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:08.121120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:08.140667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:08.214962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1009/2032] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:08.256773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:08.259134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:08.334282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88a2d0-fa24-4659-9186-f7a22e448821 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:08.338187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:08.338809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:08.339114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:08.339363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:08.349703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:08.352104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:08.362625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1024/2033] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:08.508675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:08.849918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:08.856427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:08.873330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eee971-9829-4252-abc1-f7af5bf70b92 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:08.880530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:08.890979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:08.890979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:08.890979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] PATH_INFO: `/v3/projects/871bc0e9f9424aa4add3f3f4db17b121` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:08.890979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=871bc0e9f9424aa4add3f3f4db17b121)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:08.890979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:09.118456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871bc0e9f9424aa4add3f3f4db17b121', 'request_id': 'req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.125680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871bc0e9f9424aa4add3f3f4db17b121', 'request_id': 'req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.229157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:09.268719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871bc0e9f9424aa4add3f3f4db17b121', 'request_id': 'req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.284908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] The token cache is being invalidate because project 871bc0e9f9424aa4add3f3f4db17b121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:09.285468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.286242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.288952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1010/2034] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:08 2022] DELETE /identity/v3/projects/871bc0e9f9424aa4add3f3f4db17b121 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:09.293122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:09.304354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed2db-51c3-4a08-a4a5-c63f6b09f30d service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:09.398250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:09.398250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:09.398250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:09.398837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] PATH_INFO: `/v3/users/e6f34552a8934d1482cfcec0c7c715ae` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:09.400319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f34552a8934d1482cfcec0c7c715ae)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:09.402585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:09.791280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f34552a8934d1482cfcec0c7c715ae'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.874673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f34552a8934d1482cfcec0c7c715ae'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.883485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f34552a8934d1482cfcec0c7c715ae'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.903835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f34552a8934d1482cfcec0c7c715ae'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.904500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f34552a8934d1482cfcec0c7c715ae'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:09.914169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1025/2035] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:09 2022] DELETE /identity/v3/users/e6f34552a8934d1482cfcec0c7c715ae => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:10.057670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:10.236507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:10.244895 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:10.356742 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e6b99-456d-4d7b-8d3d-6ebfa8d49b34 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:10.428629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:10.429264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:10.429746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:10.430178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:10.432291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:10.525881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:10.527517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:10.597493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:10.599442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf6580-4f65-433f-ab94-563fef9c7f7c service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:10.634559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1011/2036] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:10.706896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18307f3-d8c7-4ff1-bb3f-ac719b5a088e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:10.712551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:10.713479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:10.713766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:10.714043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] PATH_INFO: `/v3/projects/7ef98d38db134c77a7ed576ebfdcc355` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:10.715292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef98d38db134c77a7ed576ebfdcc355)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:10.717616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:10.860982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef98d38db134c77a7ed576ebfdcc355', 'request_id': 'req-85304d45-ffb1-42b1-a396-c154991b34c7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:10.867496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef98d38db134c77a7ed576ebfdcc355', 'request_id': 'req-85304d45-ffb1-42b1-a396-c154991b34c7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:11.006689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef98d38db134c77a7ed576ebfdcc355', 'request_id': 'req-85304d45-ffb1-42b1-a396-c154991b34c7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:11.040887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] The token cache is being invalidate because project 7ef98d38db134c77a7ed576ebfdcc355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:11.041334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:11.042227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-85304d45-ffb1-42b1-a396-c154991b34c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:11.044318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1026/2037] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:10 2022] DELETE /identity/v3/projects/7ef98d38db134c77a7ed576ebfdcc355 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:11.054764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.055525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.056334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.083035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:11.084449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:11.171950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1012/2038] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:11.179704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.180394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.180880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.211037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:11.212668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:11.224795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1027/2039] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:11.229226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:11.241108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5127d3-0b66-42f4-8345-e1f4f29d38f0 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:11.253354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:11.254846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.255308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.257281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.259010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:11.260385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:11.262645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:11.291417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1013/2040] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:11.296699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:11.313131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6815e-99a3-48ee-81c3-593a35bad86c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:11.323618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:11.324841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.325589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.326123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.327454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:11.332091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:11.556322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1028/2041] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:11 2022] POST /identity/v3/projects => generated 387 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:11.563747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:11.581856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccff05-c83d-4749-a8e3-adc226c402aa admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:11.675943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:11.677283 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.677644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.678069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.679828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:11.684525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:11.861441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1014/2042] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:11 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:11.866754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:11.880386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc331614-5f4d-4f88-b81d-7759a7bc973c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:11.887610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:11.895950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.896416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.896848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.897551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:11.899237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:11.911278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1029/2043] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:11.915752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:11.926782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971d1a9-019e-4afb-ab44-1915607442be admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:11.931539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32040de4-6d12-4ab7-bcc4-89a83115d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:11.932196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040de4-6d12-4ab7-bcc4-89a83115d898 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.932484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040de4-6d12-4ab7-bcc4-89a83115d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.932756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040de4-6d12-4ab7-bcc4-89a83115d898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.933405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32040de4-6d12-4ab7-bcc4-89a83115d898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:11.935255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32040de4-6d12-4ab7-bcc4-89a83115d898 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:11.947555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1015/2044] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:11.952274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:11.964855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54882dab-bdd8-4d43-85ba-9815785e57be admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:11.971527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:11.972213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:11.972588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:11.972748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] PATH_INFO: `/v3/projects/28953fafda984af497d5fd869eeb2c7f/users/6a4cf1fee5fa485f9963c0f2f6a459a0/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:11.990127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28953fafda984af497d5fd869eeb2c7f, user_id=6a4cf1fee5fa485f9963c0f2f6a459a0, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:11.993463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:12.193003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1030/2045] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:11 2022] PUT /identity/v3/projects/28953fafda984af497d5fd869eeb2c7f/users/6a4cf1fee5fa485f9963c0f2f6a459a0/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:12.201752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.202038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.202408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.247698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] MFA Rules not processed for user `6a4cf1fee5fa485f9963c0f2f6a459a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:12.251952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:12.376040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1016/2046] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:12.380455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:12.466155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7195bf1-d83a-4d81-9808-f84611e478e2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:12.472784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:12.472784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.480723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.480723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.480723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:12.480723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:12.619284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:12.629349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1031/2047] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:12 2022] POST /identity/v3/projects => generated 387 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:12.633573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:12.645806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab8d21-00c5-4bd2-b282-27cbd1691af2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:12.723453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:12.724548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.724548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.726765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.726765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:12.727625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:12.738245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:12.885808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1017/2048] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:12 2022] POST /identity/v3/users => generated 300 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:12.889814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:12.903166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595f3ba-bc76-4663-bcba-96cd2a7c04f6 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:12.910375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:12.911004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.911732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.912198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.913049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:12.915285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:12.931152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1032/2049] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:12.935157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:12.946613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bd3e-362a-409b-b103-6a1ab9e84939 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:12.955855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:12.955855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:12.955855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:12.955855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] PATH_INFO: `/v3/projects/becf7650245e4237a8227d386ec1c66c/users/ea54eb7f80ad49aba6522bfab320e5d7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:12.975110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=becf7650245e4237a8227d386ec1c66c, user_id=ea54eb7f80ad49aba6522bfab320e5d7, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:12.978651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:13.076608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1018/2050] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:12 2022] PUT /identity/v3/projects/becf7650245e4237a8227d386ec1c66c/users/ea54eb7f80ad49aba6522bfab320e5d7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:13.080716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:13.091763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30ce52-af25-4ca3-a048-97ef0dbad10e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:13.168315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de714de-e273-487c-8f54-39932605ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:13.168315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de714de-e273-487c-8f54-39932605ccf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:13.168315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de714de-e273-487c-8f54-39932605ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:13.168315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de714de-e273-487c-8f54-39932605ccf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:13.169366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de714de-e273-487c-8f54-39932605ccf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:13.171063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de714de-e273-487c-8f54-39932605ccf5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:13.183382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1033/2051] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:13.187942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:13.211077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1e07-f3db-455d-a7d3-1a9f993edef4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:13.232924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:13.234419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:13.234554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:13.234820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] PATH_INFO: `/v3/projects/becf7650245e4237a8227d386ec1c66c/users/ea54eb7f80ad49aba6522bfab320e5d7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:13.237022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=becf7650245e4237a8227d386ec1c66c, user_id=ea54eb7f80ad49aba6522bfab320e5d7, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:13.240201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:13.251486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1019/2052] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:13 2022] PUT /identity/v3/projects/becf7650245e4237a8227d386ec1c66c/users/ea54eb7f80ad49aba6522bfab320e5d7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:13.263769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:13.264318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:13.264592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:13.297382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] MFA Rules not processed for user `ea54eb7f80ad49aba6522bfab320e5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:13.298603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:13.373916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1034/2053] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:13.385985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:13.387907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:13.466365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fb457-f4bc-41af-9440-0ad91620b561 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:13.471474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390d628-4aae-44e0-8242-381b085fbd5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:13.473320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390d628-4aae-44e0-8242-381b085fbd5e service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:13.473869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390d628-4aae-44e0-8242-381b085fbd5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:13.474351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390d628-4aae-44e0-8242-381b085fbd5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:13.486087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390d628-4aae-44e0-8242-381b085fbd5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:13.488004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390d628-4aae-44e0-8242-381b085fbd5e service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:13.500401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1020/2054] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:13.576955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:13.656383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87f56e-8042-4103-aa14-b8823fbeef5c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:13.662056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:13.662963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:13.663394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:13.663833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] PATH_INFO: `/v3/users/6a4cf1fee5fa485f9963c0f2f6a459a0` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:13.665305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4cf1fee5fa485f9963c0f2f6a459a0)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:13.667501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:13.957711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4cf1fee5fa485f9963c0f2f6a459a0'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:14.052665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4cf1fee5fa485f9963c0f2f6a459a0'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:14.064096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4cf1fee5fa485f9963c0f2f6a459a0'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:14.075560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4cf1fee5fa485f9963c0f2f6a459a0'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:14.076313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4cf1fee5fa485f9963c0f2f6a459a0'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:14.086235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1035/2055] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:13 2022] DELETE /identity/v3/users/6a4cf1fee5fa485f9963c0f2f6a459a0 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:14.092774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37b037-2053-47bf-a5af-aa0296884e99 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:14.092774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37b037-2053-47bf-a5af-aa0296884e99 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:14.092774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37b037-2053-47bf-a5af-aa0296884e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:14.116256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5a37b037-2053-47bf-a5af-aa0296884e99 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:14.117737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5a37b037-2053-47bf-a5af-aa0296884e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:14.195964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1021/2056] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:14.201991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:14.204109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:14.293874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68853ae-18d8-4dbd-8407-23e1440dd5ce admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:14.298435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:14.299073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:14.299359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:14.299631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:14.314310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:14.314900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:14.327773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1036/2057] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:14.590072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:14.590587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:14.590674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:14.635633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] MFA Rules not processed for user `0e3f8c861a9c49119fc531e2a159ec9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:14.637675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:14.730288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1022/2058] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:14.747574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:14.749615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:14.871291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:14.873372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087626ca-ac0f-4eb6-a108-70c36f7ca00c service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:14.882334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:14.882334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:14.882334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:14.882334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:14.890981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:14.892885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:14.904856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1037/2059] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:14.909348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:15.230111 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:15.231543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:15.246559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22475d46-93c8-49d1-b7e0-d0eecb318e0b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:15.253419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:15.254079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:15.254360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:15.254645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] PATH_INFO: `/v3/projects/28953fafda984af497d5fd869eeb2c7f` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:15.255981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28953fafda984af497d5fd869eeb2c7f)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:15.258330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:15.487833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28953fafda984af497d5fd869eeb2c7f', 'request_id': 'req-bb770eb6-8127-4917-98d1-ce71ffbc8895'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:15.494935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28953fafda984af497d5fd869eeb2c7f', 'request_id': 'req-bb770eb6-8127-4917-98d1-ce71ffbc8895'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:15.616789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28953fafda984af497d5fd869eeb2c7f', 'request_id': 'req-bb770eb6-8127-4917-98d1-ce71ffbc8895'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:15.629593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] The token cache is being invalidate because project 28953fafda984af497d5fd869eeb2c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:15.629888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:15.640628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:15.640628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1023/2060] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:15 2022] DELETE /identity/v3/projects/28953fafda984af497d5fd869eeb2c7f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:15.643642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:15.655068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccb1b6-293c-4de8-8b7d-7667dac93e2b service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:15.723132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:15.723769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:15.724080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:15.724361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] PATH_INFO: `/v3/users/ea54eb7f80ad49aba6522bfab320e5d7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:15.725480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea54eb7f80ad49aba6522bfab320e5d7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:15.727394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:16.021379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea54eb7f80ad49aba6522bfab320e5d7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:16.325833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea54eb7f80ad49aba6522bfab320e5d7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:16.339000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea54eb7f80ad49aba6522bfab320e5d7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:16.384176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea54eb7f80ad49aba6522bfab320e5d7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:16.384975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b01012d-8590-438e-821e-e1e004d29efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea54eb7f80ad49aba6522bfab320e5d7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:16.398482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1038/2061] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:15 2022] DELETE /identity/v3/users/ea54eb7f80ad49aba6522bfab320e5d7 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:16.931920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:16.940727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:16.942496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:16.962667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770eb6-8127-4917-98d1-ce71ffbc8895 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:17.037592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:17.039263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:17.039716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:17.040776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] PATH_INFO: `/v3/projects/becf7650245e4237a8227d386ec1c66c` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:17.042200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=becf7650245e4237a8227d386ec1c66c)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:17.047050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:17.296967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'becf7650245e4237a8227d386ec1c66c', 'request_id': 'req-7814f97a-4158-442c-80d8-9a7d56a1f734'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:17.304854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'becf7650245e4237a8227d386ec1c66c', 'request_id': 'req-7814f97a-4158-442c-80d8-9a7d56a1f734'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:17.477835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'becf7650245e4237a8227d386ec1c66c', 'request_id': 'req-7814f97a-4158-442c-80d8-9a7d56a1f734'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:17.501080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] The token cache is being invalidate because project becf7650245e4237a8227d386ec1c66c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:17.501379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:17.502058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7814f97a-4158-442c-80d8-9a7d56a1f734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:17.503590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1024/2062] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:16 2022] DELETE /identity/v3/projects/becf7650245e4237a8227d386ec1c66c => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:17.508725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:17.509206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:17.509375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:17.538788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:17.540578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:17.634875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1039/2063] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:17.638597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:17.638597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:17.638597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:17.660797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:17.662052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:17.675417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1025/2064] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:17.676734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:17.687858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b033a-8faf-4378-a19e-9d5af7b54ef8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:17.693158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:17.693921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:17.694273 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:17.694557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:17.695537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:17.695650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:17.697412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:17.714423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1040/2065] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:17.715870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:17.729076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4a29a-e7b1-4539-8e84-26c256b825ee None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:17.735247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:17.735443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:17.735725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:17.747564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:17.747564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:17.747564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:17.947352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1026/2066] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:17 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:17.951714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:17.964603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d01e8-76b2-43a8-ae0c-05b272d639ae admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.034381 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.035035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.035362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.035644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.036631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.038618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:18.102393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:18.184185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1041/2067] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:17 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:18.195221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:18.227224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b03c6-55de-4b5b-92b9-84768194206c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.240126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.242577 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.243185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.246689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.249860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.254310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:18.277633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1027/2068] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:18.281817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:18.316723 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6b68c-5feb-40dd-a140-49b4899111e2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.330691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e3126-18a8-4303-8c31-e7c13f2a0ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.331545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e3126-18a8-4303-8c31-e7c13f2a0ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.331798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e3126-18a8-4303-8c31-e7c13f2a0ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.332147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e3126-18a8-4303-8c31-e7c13f2a0ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.332822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e3126-18a8-4303-8c31-e7c13f2a0ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.334680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e3126-18a8-4303-8c31-e7c13f2a0ef5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:18.350764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1042/2069] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:18.361058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:18.373473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c81a7-17e5-430c-b3a9-57b6adcc9ca4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.390359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.391069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.391353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.391632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] PATH_INFO: `/v3/projects/faaa73fc81a945b691e8945734ca1849/users/3a44954b580e4d5e92eaca8a051578e8/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.431607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faaa73fc81a945b691e8945734ca1849, user_id=3a44954b580e4d5e92eaca8a051578e8, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.434390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:18.521050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1028/2070] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:18 2022] PUT /identity/v3/projects/faaa73fc81a945b691e8945734ca1849/users/3a44954b580e4d5e92eaca8a051578e8/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:18.533104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163a205-9bdd-4f32-890f-901f126da73b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.533680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163a205-9bdd-4f32-890f-901f126da73b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.533680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163a205-9bdd-4f32-890f-901f126da73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.568799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b163a205-9bdd-4f32-890f-901f126da73b None None] MFA Rules not processed for user `3a44954b580e4d5e92eaca8a051578e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:18.570062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b163a205-9bdd-4f32-890f-901f126da73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:18.683928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1043/2071] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:18.685167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:18.757260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f0f10-8e2f-4d33-ae37-3fb614add3e8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.761982 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.762598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.762891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.763156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.763790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.765826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:18.865328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.865688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.865967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.882812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1029/2072] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:18 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:18.886236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] MFA Rules not processed for user `0e3f8c861a9c49119fc531e2a159ec9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:18.887463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:18.892225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:18.952650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1044/2073] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:18.966127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59498ab-577b-403e-ad12-f711c4ce4f2e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:18.973818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:18.975993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:18.975993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:18.975993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:18.975993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:18.977755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:19.104165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1030/2074] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:18 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:19.109963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:19.123611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af8af4-443a-4641-83c8-88751011f2c0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:19.137239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:19.137935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:19.138345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:19.138721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:19.139591 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:19.142145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:19.158542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1045/2075] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:19.162782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:19.175603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9d1d-7de8-45cd-b304-fb7f9fa77c4a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:19.181983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:19.182657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:19.182937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:19.183242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] PATH_INFO: `/v3/projects/75ca92c9ab4b414e82be23c641cca9cd/users/e1eaa484bc084835b1573a386884b357/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:19.200207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ca92c9ab4b414e82be23c641cca9cd, user_id=e1eaa484bc084835b1573a386884b357, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:19.202931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:19.263628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:19.350191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1031/2076] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:19 2022] PUT /identity/v3/projects/75ca92c9ab4b414e82be23c641cca9cd/users/e1eaa484bc084835b1573a386884b357/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:19.354457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:19.365095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73b200-89df-4529-b836-380e65f63728 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:19.431006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467bde1-8752-4ec1-bd41-a97a21abcff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:19.431865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467bde1-8752-4ec1-bd41-a97a21abcff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:19.432365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467bde1-8752-4ec1-bd41-a97a21abcff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:19.432743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467bde1-8752-4ec1-bd41-a97a21abcff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:19.433505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467bde1-8752-4ec1-bd41-a97a21abcff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:19.435372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467bde1-8752-4ec1-bd41-a97a21abcff9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:19.449446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1046/2077] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:19.450927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:19.462401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52167975-ec8d-4c5c-91f2-3d445d8f5dbf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:19.467293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:19.468126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:19.468910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:19.474195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] PATH_INFO: `/v3/projects/75ca92c9ab4b414e82be23c641cca9cd/users/e1eaa484bc084835b1573a386884b357/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:19.474195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ca92c9ab4b414e82be23c641cca9cd, user_id=e1eaa484bc084835b1573a386884b357, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:19.474195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:19.486443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1032/2078] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:19 2022] PUT /identity/v3/projects/75ca92c9ab4b414e82be23c641cca9cd/users/e1eaa484bc084835b1573a386884b357/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:19.497070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:19.497528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:19.497816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:19.530940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] MFA Rules not processed for user `e1eaa484bc084835b1573a386884b357`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:19.532330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:19.613923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1047/2079] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:19.627282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:19.630117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:19.697876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f256f-78d3-4920-b910-e5eb6b437419 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:19.701435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:19.702060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:19.702399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:19.702644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:19.712574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:19.713928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:19.725104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1033/2080] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:21.376218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:21.395099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db265cf3-c24b-4b98-9405-e6315f8bd07c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:21.408098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1eaa484bc084835b1573a386884b357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75ca92c9ab4b414e82be23c641cca9cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:21.409315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.409651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.410353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.411503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:21.412128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:21.415980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:21.432456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1048/2081] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:21.433630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:21.446848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db33dcf-89a8-4f40-9a40-1a823c821da0 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:21.460203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1eaa484bc084835b1573a386884b357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75ca92c9ab4b414e82be23c641cca9cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:21.460720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.461012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.461287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.462032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:21.464298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:21.642813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1034/2082] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:21 2022] POST /identity/v3/projects => generated 353 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:21.647865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:21.659127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d79e68-b7bd-4a83-946f-2ed5ccd883e2 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:21.750274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1eaa484bc084835b1573a386884b357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75ca92c9ab4b414e82be23c641cca9cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:21.751036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.751399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.751747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.752592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:21.752971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:21.754685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:21.768770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1049/2083] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:28:21.810176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:21.821967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac5b63-5daf-4241-8495-fb11656f56f3 tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:21.830263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1eaa484bc084835b1573a386884b357', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75ca92c9ab4b414e82be23c641cca9cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:21.831016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:21.831302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:21.831615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] PATH_INFO: `/v3/projects/e8d3d2d79e194bd7a3b611f90d0d2b2b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:21.833087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e8d3d2d79e194bd7a3b611f90d0d2b2b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:21.834981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:22.008924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d3d2d79e194bd7a3b611f90d0d2b2b', 'request_id': 'req-9a975385-4171-4895-93f7-ef1334cdaf6b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:22.017371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d3d2d79e194bd7a3b611f90d0d2b2b', 'request_id': 'req-9a975385-4171-4895-93f7-ef1334cdaf6b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:22.143115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d3d2d79e194bd7a3b611f90d0d2b2b', 'request_id': 'req-9a975385-4171-4895-93f7-ef1334cdaf6b'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:22.159417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] The token cache is being invalidate because project e8d3d2d79e194bd7a3b611f90d0d2b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:22.159713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:22.160431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:22.161950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1035/2084] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:21 2022] DELETE /identity/v3/projects/e8d3d2d79e194bd7a3b611f90d0d2b2b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:22.314845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:23.245679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:23.251257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:23.339266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:23.341138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:23.400731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae758e-a6e4-4dfb-8baa-ab874f7da78a tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:23.455144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a975385-4171-4895-93f7-ef1334cdaf6b tempest-VolumeQuotaClassesTest-774658529 tempest-VolumeQuotaClassesTest-774658529-project-admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:23.461148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:23.463167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:23.463635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:23.464323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] PATH_INFO: `/v3/users/b5b547711d184917a2d5616d95fdf3b8` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:23.467204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b547711d184917a2d5616d95fdf3b8)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:23.470992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:23.497528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:23.522588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:23.523557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:23.523805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:23.524294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] PATH_INFO: `/v3/users/3a44954b580e4d5e92eaca8a051578e8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:23.525685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a44954b580e4d5e92eaca8a051578e8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:23.527643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:23.793641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b547711d184917a2d5616d95fdf3b8'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:23.794602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b547711d184917a2d5616d95fdf3b8'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:23.812870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b547711d184917a2d5616d95fdf3b8'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:23.814917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b547711d184917a2d5616d95fdf3b8'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:23.826450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-cc266c64-c824-4c3b-8343-d7dc8acd14d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b547711d184917a2d5616d95fdf3b8'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:23.924738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a44954b580e4d5e92eaca8a051578e8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:23.951305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1036/2085] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:23 2022] DELETE /identity/v3/users/b5b547711d184917a2d5616d95fdf3b8 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:23.956513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:23.956996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:23.957503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:23.985596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:23.987009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.042107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a44954b580e4d5e92eaca8a051578e8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:24.051988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a44954b580e4d5e92eaca8a051578e8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:24.078743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a44954b580e4d5e92eaca8a051578e8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:24.084326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0e0c4a55-bf65-4501-8c97-80d071db5b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a44954b580e4d5e92eaca8a051578e8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:24.088430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1050/2086] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:23 2022] DELETE /identity/v3/users/3a44954b580e4d5e92eaca8a051578e8 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:24.091756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1037/2087] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:24.099092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:24.099459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:24.099805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:24.110230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:24.112639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.133913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:24.135468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.223965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38c993-ee80-42a1-ba51-d7a6b62d76b9 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:24.268246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1051/2088] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:24.282031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:24.283663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.306274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:24.307054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:24.308052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:24.308052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:24.310599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:24.335790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:24.340894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:24.358848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1038/2089] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:24.375557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78afa60-b545-4269-bde7-ff311d777687 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:24.380587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:24.380587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:24.380944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:24.381141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:24.394733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:24.396474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:24.407609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1052/2090] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:25.088219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:25.090101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:25.110050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709965c-6f16-4d2c-adf4-9ddf8389e8f0 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:25.117149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:25.119597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:25.120069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:25.120537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] PATH_INFO: `/v3/projects/faaa73fc81a945b691e8945734ca1849` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:25.121732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faaa73fc81a945b691e8945734ca1849)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:25.123840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:25.142554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:25.144406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:25.160569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c667b-2744-4bc8-b7a8-598301476d31 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:25.166155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:25.167175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:25.167619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:25.168263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] PATH_INFO: `/v3/projects/f7bcfe31b87d44ef9e0106ecd4d68bad` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:25.169793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7bcfe31b87d44ef9e0106ecd4d68bad)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:25.172668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:25.255671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faaa73fc81a945b691e8945734ca1849', 'request_id': 'req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.262367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faaa73fc81a945b691e8945734ca1849', 'request_id': 'req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.321754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7bcfe31b87d44ef9e0106ecd4d68bad', 'request_id': 'req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.325944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faaa73fc81a945b691e8945734ca1849', 'request_id': 'req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.332799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:25.338438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7bcfe31b87d44ef9e0106ecd4d68bad', 'request_id': 'req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.340822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] The token cache is being invalidate because project faaa73fc81a945b691e8945734ca1849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:25.341118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.341786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.347713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1039/2091] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:25 2022] DELETE /identity/v3/projects/faaa73fc81a945b691e8945734ca1849 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:25.348984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:25.350697 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:25.436245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fecdd-6ea8-4b54-b459-8aa5e684e845 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:25.463813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7bcfe31b87d44ef9e0106ecd4d68bad', 'request_id': 'req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.482686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] The token cache is being invalidate because project f7bcfe31b87d44ef9e0106ecd4d68bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:25.483046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.484476 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.487215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1053/2092] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:25 2022] DELETE /identity/v3/projects/f7bcfe31b87d44ef9e0106ecd4d68bad => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:25.493816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:25.495707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:25.502879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:25.503651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:25.504155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:25.504531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] PATH_INFO: `/v3/users/e1eaa484bc084835b1573a386884b357` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:25.506051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1eaa484bc084835b1573a386884b357)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:25.508218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:25.511861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dedaeb-a379-476c-a9fe-9d980c5b5b42 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:25.518438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:25.519238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:25.519635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:25.519940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] PATH_INFO: `/v3/users/66a56512959e495eab4686e66a47eaec` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:25.521447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a56512959e495eab4686e66a47eaec)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:25.523425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:25.768190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a56512959e495eab4686e66a47eaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.791438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1eaa484bc084835b1573a386884b357'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.792120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1eaa484bc084835b1573a386884b357'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.805405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1eaa484bc084835b1573a386884b357'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.806177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1eaa484bc084835b1573a386884b357'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.813743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1eaa484bc084835b1573a386884b357'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.869279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a56512959e495eab4686e66a47eaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.881010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a56512959e495eab4686e66a47eaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.904234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a56512959e495eab4686e66a47eaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.906054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a56512959e495eab4686e66a47eaec'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:25.914068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1054/2093] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:25 2022] DELETE /identity/v3/users/66a56512959e495eab4686e66a47eaec => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:25.931539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1040/2094] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:25 2022] DELETE /identity/v3/users/e1eaa484bc084835b1573a386884b357 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:26.227964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:26.584572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:26.586537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:26.593350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:26.595184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:26.676755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede270d0-6390-49b2-8727-0d32c783fabb admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:26.681495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:26.682140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:26.682419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:26.682691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] PATH_INFO: `/v3/projects/75ca92c9ab4b414e82be23c641cca9cd` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:26.685395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ca92c9ab4b414e82be23c641cca9cd)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:26.686390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce96b3-2bad-4226-8d14-402549528ec1 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:26.687788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:26.691990 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:26.692724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:26.693127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:26.695532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] PATH_INFO: `/v3/projects/39721950a9ca4e32a9016ed501e68da2` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:26.696921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39721950a9ca4e32a9016ed501e68da2)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:26.698997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:26.822811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ca92c9ab4b414e82be23c641cca9cd', 'request_id': 'req-abb6e47c-e389-43f5-a051-428ccad94ab4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.829171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ca92c9ab4b414e82be23c641cca9cd', 'request_id': 'req-abb6e47c-e389-43f5-a051-428ccad94ab4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.860689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39721950a9ca4e32a9016ed501e68da2', 'request_id': 'req-ec683615-89db-4e87-bf83-d80ef9110535'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.866780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39721950a9ca4e32a9016ed501e68da2', 'request_id': 'req-ec683615-89db-4e87-bf83-d80ef9110535'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.965784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ca92c9ab4b414e82be23c641cca9cd', 'request_id': 'req-abb6e47c-e389-43f5-a051-428ccad94ab4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.978097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] The token cache is being invalidate because project 75ca92c9ab4b414e82be23c641cca9cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:26.978395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.979039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-abb6e47c-e389-43f5-a051-428ccad94ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:26.980574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1055/2095] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:26 2022] DELETE /identity/v3/projects/75ca92c9ab4b414e82be23c641cca9cd => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:26.985614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca093c4-3cd0-4492-bbd3-4ed9ada801c3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:26.985936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca093c4-3cd0-4492-bbd3-4ed9ada801c3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:26.986209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca093c4-3cd0-4492-bbd3-4ed9ada801c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.006560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9ca093c4-3cd0-4492-bbd3-4ed9ada801c3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:27.007768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9ca093c4-3cd0-4492-bbd3-4ed9ada801c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:27.052045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39721950a9ca4e32a9016ed501e68da2', 'request_id': 'req-ec683615-89db-4e87-bf83-d80ef9110535'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.064972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] The token cache is being invalidate because project 39721950a9ca4e32a9016ed501e68da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:27.065348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.066096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.067696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1041/2096] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:26 2022] DELETE /identity/v3/projects/39721950a9ca4e32a9016ed501e68da2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:27.071494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.072481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1056/2097] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:27.073147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:27.081573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.082095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.082377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.087069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec683615-89db-4e87-bf83-d80ef9110535 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.092209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.093481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.093816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.094109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] PATH_INFO: `/v3/users/1d94b485940b498dab092551742d97d3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.095450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d94b485940b498dab092551742d97d3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.097521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.102949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:27.104238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:27.112243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1057/2098] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:27.116731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.127872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ca4ce-b656-48b1-8604-847f672f74d1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.147502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.147502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.147502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.147502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.147859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:27.147859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.147859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.159410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1058/2099] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:28:27.164114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.183958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354d62-9e86-44a6-8131-6a171192c636 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.183958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.183958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.183958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.187586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.187586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.187586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.288104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1059/2100] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:27 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:27.292475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.349712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d94b485940b498dab092551742d97d3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.350357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d94b485940b498dab092551742d97d3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.356143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbe0af-f84d-4985-bba6-b90bd5096b9c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.361456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.362069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.362354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.362629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.363297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.363433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d94b485940b498dab092551742d97d3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.364069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d94b485940b498dab092551742d97d3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.365269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.371061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d94b485940b498dab092551742d97d3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:27.560582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1042/2101] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:27 2022] DELETE /identity/v3/users/1d94b485940b498dab092551742d97d3 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:27.561161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1060/2102] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:27 2022] POST /identity/v3/users => generated 295 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:27.565489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.567016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:27.636328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5360d7-29df-4e66-b2bf-d3409aedfdc8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.641439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.642063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.642345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.642617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.643284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.644940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.659491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1043/2103] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:27.660734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.676310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1122e2d-f6df-438f-8dab-bfc42081128a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.687178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406f9e-b781-4e33-879d-08c0d8322508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.688475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406f9e-b781-4e33-879d-08c0d8322508 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.688784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406f9e-b781-4e33-879d-08c0d8322508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.689056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406f9e-b781-4e33-879d-08c0d8322508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.689710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa406f9e-b781-4e33-879d-08c0d8322508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.692861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa406f9e-b781-4e33-879d-08c0d8322508 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.703739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1061/2104] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:27.707735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:27.718709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa45d02-072f-470d-91fc-e15d44029e60 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:27.723348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:27.724060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.724351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.724704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] PATH_INFO: `/v3/projects/a6e99959d864481d9dcae0807fb4986b/users/162cfe7bd4f141548b393cc8644c0181/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.742293 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e99959d864481d9dcae0807fb4986b, user_id=162cfe7bd4f141548b393cc8644c0181, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:27.745213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:27.783353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:27.891538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1044/2105] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:27 2022] PUT /identity/v3/projects/a6e99959d864481d9dcae0807fb4986b/users/162cfe7bd4f141548b393cc8644c0181/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:27.903373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:27.903817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:27.904139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:27.936274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] MFA Rules not processed for user `162cfe7bd4f141548b393cc8644c0181`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:27.937589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:28.008876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1062/2106] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:28.013697 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.082523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753c176-6bcc-4120-8f7e-f50e4070a895 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.087711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.088393 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.088675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.088949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.089631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.091672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.156498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.158413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:28.175698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9da64-0033-4c4c-944d-5a36d9dfb43d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.181481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.182434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.182873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.185431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] PATH_INFO: `/v3/projects/37cfb839e44c46a385d08c8b2677e7fc` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.196808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cfb839e44c46a385d08c8b2677e7fc)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.198878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.237649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1045/2107] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:28 2022] POST /identity/v3/projects => generated 389 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:28.241800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.325535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24223e-34b6-41d9-9e7a-181e5e2fe996 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.332637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.333365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.334462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.334462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.334462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.337726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.441131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cfb839e44c46a385d08c8b2677e7fc', 'request_id': 'req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:28.448903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cfb839e44c46a385d08c8b2677e7fc', 'request_id': 'req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:28.507417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1046/2108] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:28 2022] POST /identity/v3/users => generated 301 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:28.509624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.521246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a7101-21e6-4447-9f12-bc05b9f5165e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.591608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.592312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.592660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.593512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.594264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.596022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.611189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1047/2109] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:28.612050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cfb839e44c46a385d08c8b2677e7fc', 'request_id': 'req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:28.612636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.624846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426cdd6-da8e-47c9-acb5-d425aa8a17c3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.626106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] The token cache is being invalidate because project 37cfb839e44c46a385d08c8b2677e7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:28.626422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:28.628423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-fac73d1a-a3cf-48ef-96d2-fd75aa14277b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:28.628752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1063/2110] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:28 2022] DELETE /identity/v3/projects/37cfb839e44c46a385d08c8b2677e7fc => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:28.630348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.631160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.631292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.631570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] PATH_INFO: `/v3/projects/82933d9ce639498fbf72783806f33a31/users/1bbcdb250b9549b497b0ef1055a2327c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.633859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb50de-313b-45e1-a336-3f527e316ea1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.634155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb50de-313b-45e1-a336-3f527e316ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.634436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb50de-313b-45e1-a336-3f527e316ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.650523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82933d9ce639498fbf72783806f33a31, user_id=1bbcdb250b9549b497b0ef1055a2327c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.653311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.659493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-31cb50de-313b-45e1-a336-3f527e316ea1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:28.660992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-31cb50de-313b-45e1-a336-3f527e316ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:28.670806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1064/2111] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:28.681252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.681252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.681252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.707672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:28.709189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:28.722334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1065/2112] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:28.725339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.744111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8254d4-4cd0-4f6d-82cc-7d5167cb95d8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.752563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.753222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.753505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.753777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.754437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:28.754882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.756584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.779937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1066/2113] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:28:28.779937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.789040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d4f7f-b781-43aa-b10d-d6d19856d1cd admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.794640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.795165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.795446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.795790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.796950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.798994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.808527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1048/2114] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:28 2022] PUT /identity/v3/projects/82933d9ce639498fbf72783806f33a31/users/1bbcdb250b9549b497b0ef1055a2327c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:28.821496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.823760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:28.925948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead09b37-e343-4e42-8af5-53a12e889297 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:28.933954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:28.934638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:28.934927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:28.935203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:28.935868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:28.948013 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:28.958359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1067/2115] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:28 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:28.964637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:28.973868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:28.984693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1049/2116] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:28.988972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.005958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d73ad-61b0-49b8-8158-94180cbf2074 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.058265 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b86c0-1aae-449d-ab69-e627b720df80 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.064426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.065327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.072377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.076566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.076566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.076566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.081598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b6bb5-7c42-409b-8440-efee619d441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.082425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b6bb5-7c42-409b-8440-efee619d441e admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.082425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b6bb5-7c42-409b-8440-efee619d441e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.082885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b6bb5-7c42-409b-8440-efee619d441e admin admin] PATH_INFO: `/v3/projects/82933d9ce639498fbf72783806f33a31/users/1bbcdb250b9549b497b0ef1055a2327c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.085041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b6bb5-7c42-409b-8440-efee619d441e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82933d9ce639498fbf72783806f33a31, user_id=1bbcdb250b9549b497b0ef1055a2327c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.088197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b6bb5-7c42-409b-8440-efee619d441e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.107331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1050/2117] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:28 2022] PUT /identity/v3/projects/82933d9ce639498fbf72783806f33a31/users/1bbcdb250b9549b497b0ef1055a2327c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:29.118144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.118474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.118718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.161091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] MFA Rules not processed for user `1bbcdb250b9549b497b0ef1055a2327c`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:29.162498 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:29.237006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1068/2118] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:28 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:29.243127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.262951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29161cae-c39d-428a-a348-dba4149808d6 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.267587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1051/2119] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:29.286942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.288419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:29.338739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.339690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.339976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.340257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.341083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.342684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.359828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1069/2120] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:29.360623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.362127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:29.372090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94660293-ad49-4e51-8f5d-9727012124b5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.380508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.381260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.381594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.381925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] PATH_INFO: `/v3/users/870103e49ddd46c28fb4dff1b24d41f6` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.383187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=870103e49ddd46c28fb4dff1b24d41f6)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.385312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.431294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b080765-a59c-43b6-9e88-008561ad7b0b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.435786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.437119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.437401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.437670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.448782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.450246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.461513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1070/2121] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:29.462638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.475282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c11ffa-ac68-48a7-b33c-170f2486cbcc service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.481487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.482632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.483121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.483623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.484737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.487505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.506113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1071/2122] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:29.511939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.524736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f430c2-63e0-40c5-9499-971c217f6a5b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.529627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.530342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.530612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.530889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] PATH_INFO: `/v3/projects/696bce9f557e4ebd805e82474e3ecf44/users/aef3edf8fa8844d2919c8faaa2e47804/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.549204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=696bce9f557e4ebd805e82474e3ecf44, user_id=aef3edf8fa8844d2919c8faaa2e47804, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.552148 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.562280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:29.634849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870103e49ddd46c28fb4dff1b24d41f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:29.636676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870103e49ddd46c28fb4dff1b24d41f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:29.657576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870103e49ddd46c28fb4dff1b24d41f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:29.659357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870103e49ddd46c28fb4dff1b24d41f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:29.668504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0ac74b30-0eec-47dc-b035-edd18785a9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870103e49ddd46c28fb4dff1b24d41f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:29.686177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1072/2123] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:29 2022] PUT /identity/v3/projects/696bce9f557e4ebd805e82474e3ecf44/users/aef3edf8fa8844d2919c8faaa2e47804/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:29.688034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:29.690436 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:29.776944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20911d3c-1356-4ce6-8191-d7cd776ee98a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:29.786510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:29.787722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.788083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.788372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] PATH_INFO: `/v3/users/162cfe7bd4f141548b393cc8644c0181` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.789717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162cfe7bd4f141548b393cc8644c0181)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:29.789972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1052/2124] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:29 2022] DELETE /identity/v3/users/870103e49ddd46c28fb4dff1b24d41f6 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:29.792118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:29.794698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a313-9018-4ac4-a814-32bfd9adaa15 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.796937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a313-9018-4ac4-a814-32bfd9adaa15 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.798727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a313-9018-4ac4-a814-32bfd9adaa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.836508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5cb5a313-9018-4ac4-a814-32bfd9adaa15 None None] MFA Rules not processed for user `aef3edf8fa8844d2919c8faaa2e47804`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:29.838153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5cb5a313-9018-4ac4-a814-32bfd9adaa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:29.919141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1053/2125] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:29.924456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:29.924456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:29.924456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:29.946242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:29.947744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.004942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162cfe7bd4f141548b393cc8644c0181'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:30.033396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1054/2126] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:30.034539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:30.040717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.115202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041104d-efa3-40c2-8d5e-fef68bbbd008 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:30.117077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162cfe7bd4f141548b393cc8644c0181'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:30.120536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:30.121401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:30.121906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:30.122196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:30.132882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:30.134543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:30.136153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162cfe7bd4f141548b393cc8644c0181'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:30.148892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1055/2127] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:30.156938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:30.158232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162cfe7bd4f141548b393cc8644c0181'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:30.158827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.159161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-1f646e56-c6ca-4cdd-858a-ea7cc7b77a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162cfe7bd4f141548b393cc8644c0181'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:30.172564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1073/2128] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:29 2022] DELETE /identity/v3/users/162cfe7bd4f141548b393cc8644c0181 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:30.176921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:30.177210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:30.177496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:30.205464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:30.206729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.239280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c6e2d-e029-446b-986f-66a0f2324359 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:30.283318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1074/2129] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:30.303331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:30.307112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.310282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:30.312474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:30.314448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:30.314733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:30.318837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:30.335132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:30.336440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b66908-b992-4928-b2d0-213ab0ca53f2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:30.336953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:30.352479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1056/2130] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:30.355123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:30.355872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:30.356278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:30.356558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:30.373322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:30.375471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:30.388902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1075/2131] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:31.124318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:31.125954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:31.138733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf62063-b343-44eb-89cf-e7b895d01c23 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:31.143681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:31.144385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.144731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.145083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] PATH_INFO: `/v3/projects/f6ccc70ac12444558aa2e6f4b606573a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.146353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ccc70ac12444558aa2e6f4b606573a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:31.148329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:31.258776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:31.261166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:31.275739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452963d2-8594-478a-ae81-139b2cfd539e service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:31.284231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:31.284359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.284614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.284860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] PATH_INFO: `/v3/projects/a6e99959d864481d9dcae0807fb4986b` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.286972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e99959d864481d9dcae0807fb4986b)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:31.289179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:31.386034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ccc70ac12444558aa2e6f4b606573a', 'request_id': 'req-62a83e54-fe36-4196-a575-19cc23c2f4dc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.393799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ccc70ac12444558aa2e6f4b606573a', 'request_id': 'req-62a83e54-fe36-4196-a575-19cc23c2f4dc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.571689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e99959d864481d9dcae0807fb4986b', 'request_id': 'req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.577941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e99959d864481d9dcae0807fb4986b', 'request_id': 'req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.595411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ccc70ac12444558aa2e6f4b606573a', 'request_id': 'req-62a83e54-fe36-4196-a575-19cc23c2f4dc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.596218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:31.610132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] The token cache is being invalidate because project f6ccc70ac12444558aa2e6f4b606573a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:31.611328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.611328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-62a83e54-fe36-4196-a575-19cc23c2f4dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.613408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1057/2132] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:31 2022] DELETE /identity/v3/projects/f6ccc70ac12444558aa2e6f4b606573a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:31.619263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb5aa-5fe7-472e-8dd0-e0ed7ef0aa01 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.619653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb5aa-5fe7-472e-8dd0-e0ed7ef0aa01 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.620211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb5aa-5fe7-472e-8dd0-e0ed7ef0aa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.644659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-234cb5aa-5fe7-472e-8dd0-e0ed7ef0aa01 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:31.645937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-234cb5aa-5fe7-472e-8dd0-e0ed7ef0aa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:31.698121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:31.732483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1058/2133] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:31.736853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e99959d864481d9dcae0807fb4986b', 'request_id': 'req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.741832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.743254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.743254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.750445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] The token cache is being invalidate because project a6e99959d864481d9dcae0807fb4986b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:31.750735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.751427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:31.753365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1076/2134] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:31 2022] DELETE /identity/v3/projects/a6e99959d864481d9dcae0807fb4986b => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:31.757024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:31.758398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:31.770830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:31.772227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:31.773808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27490cb-aeb9-42e6-bcda-3e8ba7d5bd97 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:31.778496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:31.779175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.779461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.779744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] PATH_INFO: `/v3/users/1bbcdb250b9549b497b0ef1055a2327c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.781668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bbcdb250b9549b497b0ef1055a2327c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:31.787933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1059/2135] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:31.788152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:31.791869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:31.809377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a604cc-c4ce-4989-98a1-63ffad6bf5c8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:31.818491 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:31.819219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.823523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.823523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.823523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:31.823523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:31.823523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:31.834853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1060/2136] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:28:31.839858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:31.850718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334309ec-2592-4c9d-b7d0-49e46cb4d579 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:31.859576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:31.861028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:31.861028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:31.861028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:31.861738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:31.863323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:32.013435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1061/2137] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:31 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:32.018867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.033203 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bbcdb250b9549b497b0ef1055a2327c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:32.083947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035953a2-b24b-4e7c-8ce1-d183bb99c60d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.089095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.089760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.089989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.090257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.090894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:32.092932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:32.127230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bbcdb250b9549b497b0ef1055a2327c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:32.156958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bbcdb250b9549b497b0ef1055a2327c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:32.171537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bbcdb250b9549b497b0ef1055a2327c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:32.172467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bbcdb250b9549b497b0ef1055a2327c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:32.182420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1077/2138] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:31 2022] DELETE /identity/v3/users/1bbcdb250b9549b497b0ef1055a2327c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:32.195271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1062/2139] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:32 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:32.196791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.198817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:32.287186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1bc4e8-0d2a-4ec9-bdcf-a32f768c6816 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.294236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.294890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.295215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.296214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.296955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:32.298620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:32.313342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1078/2140] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:32.314821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.325302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097ecfb-5020-4b36-a3d8-647577b0969a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.332723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b457f-ca77-423d-a2c9-1a98acefb92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.333350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b457f-ca77-423d-a2c9-1a98acefb92e admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.333629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b457f-ca77-423d-a2c9-1a98acefb92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.333904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b457f-ca77-423d-a2c9-1a98acefb92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.334560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b457f-ca77-423d-a2c9-1a98acefb92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:32.339372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b457f-ca77-423d-a2c9-1a98acefb92e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:32.348470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1063/2141] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:32.355474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.374437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8d43-30cf-45f6-9477-138dacc4869d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.384710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.385465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.386524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.386524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] PATH_INFO: `/v3/projects/52ac0b73cab4494ab39d80f06c54e9da/users/2e02c6c1c3bc49e8a63f0533c3ea5391/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.412409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ac0b73cab4494ab39d80f06c54e9da, user_id=2e02c6c1c3bc49e8a63f0533c3ea5391, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:32.417003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:32.549916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1079/2142] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:32 2022] PUT /identity/v3/projects/52ac0b73cab4494ab39d80f06c54e9da/users/2e02c6c1c3bc49e8a63f0533c3ea5391/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:32.555209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.555501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.555781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.578240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:32.579478 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:32.653096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1064/2143] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:32.664689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.666637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:32.736235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1c04-ebd6-4b84-990e-2b5eeb59ea8f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.745032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd9aa2-1dcd-4922-b7e6-6d615591e514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.745675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd9aa2-1dcd-4922-b7e6-6d615591e514 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.745958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd9aa2-1dcd-4922-b7e6-6d615591e514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.746237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd9aa2-1dcd-4922-b7e6-6d615591e514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.759086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd9aa2-1dcd-4922-b7e6-6d615591e514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:32.760585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd9aa2-1dcd-4922-b7e6-6d615591e514 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:32.771305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1080/2144] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:32.801739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:32.811338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:32.818507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcef8b-ead0-4840-8e2f-86e5019f2525 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:32.825118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:32.826099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:32.826459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:32.827359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] PATH_INFO: `/v3/projects/82933d9ce639498fbf72783806f33a31` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:32.828740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=82933d9ce639498fbf72783806f33a31)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:32.830819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.075583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82933d9ce639498fbf72783806f33a31', 'request_id': 'req-d66d726b-668e-4688-add0-b984c99a75ba'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:33.083700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82933d9ce639498fbf72783806f33a31', 'request_id': 'req-d66d726b-668e-4688-add0-b984c99a75ba'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:33.211089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82933d9ce639498fbf72783806f33a31', 'request_id': 'req-d66d726b-668e-4688-add0-b984c99a75ba'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:33.224451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] The token cache is being invalidate because project 82933d9ce639498fbf72783806f33a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:33.224721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:33.225479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d66d726b-668e-4688-add0-b984c99a75ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:33.227124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1065/2145] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:32 2022] DELETE /identity/v3/projects/82933d9ce639498fbf72783806f33a31 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:33.232512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.232805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.233120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.254146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:33.255531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:33.321097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1081/2146] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:33.330910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.333304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.333304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.352912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:33.354137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:33.362969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1066/2147] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:33.367690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.381234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4f13-076e-4a47-9cf7-2f4d9f81e9b7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.391184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.391813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.392125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.392694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.394333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:33.394640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.396369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.413647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1082/2148] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:33.415123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.426718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5bd7b-6532-4aea-af4c-1f43b7d9eb3a None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.430674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.431304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.431682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.431989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.432685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.434718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.542709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1067/2149] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:33 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:33.543825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.555303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bd95c-96c3-4b76-a950-3128d1301717 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.616586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.617267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.617492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.617771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.618786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.620503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.800223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1083/2150] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:33 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:33.803975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.816109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a4adc-520a-4495-be19-102ed8a937eb admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.823337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.824223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.824756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.825218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.826204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.828152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.843225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1068/2151] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:33.848549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.862865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f2e5-f396-4b4e-8b6b-a6b9d6e45a19 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.871000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.871000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.871240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.871717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.872635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.874230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.887134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1084/2152] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:33.893245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.913596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdbc9f-0107-4613-b52f-d894789dc7f7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.922251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c598b77-f542-4d9a-a23c-f99d835b8b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.923466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c598b77-f542-4d9a-a23c-f99d835b8b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.923779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c598b77-f542-4d9a-a23c-f99d835b8b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.924100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c598b77-f542-4d9a-a23c-f99d835b8b28 admin admin] PATH_INFO: `/v3/projects/0d02e1d5a80e4ee7b8dcd76f7024b147/users/0c308dbb461a48848ff8820889149c52/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.942689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:33.946089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:33.947073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c598b77-f542-4d9a-a23c-f99d835b8b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d02e1d5a80e4ee7b8dcd76f7024b147, user_id=0c308dbb461a48848ff8820889149c52, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.950022 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c598b77-f542-4d9a-a23c-f99d835b8b28 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:33.970631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524652-c834-48fe-94f6-ae5c70def7ce admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:33.981471 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:33.982235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:33.982510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:33.982792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] PATH_INFO: `/v3/users/aef3edf8fa8844d2919c8faaa2e47804` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:33.984680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef3edf8fa8844d2919c8faaa2e47804)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:33.986727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:34.051608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1069/2153] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:33 2022] PUT /identity/v3/projects/0d02e1d5a80e4ee7b8dcd76f7024b147/users/0c308dbb461a48848ff8820889149c52/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:34.057342 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:34.057623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:34.057946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:34.078099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:34.079425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:34.165142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1070/2154] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:34.197065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:34.198535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:34.248261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef3edf8fa8844d2919c8faaa2e47804'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:34.278471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6f7ea-919e-49ac-b3c7-f19896134999 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:34.282769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eef25a-54b6-4bb5-a863-6fe17ec58af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:34.283413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eef25a-54b6-4bb5-a863-6fe17ec58af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:34.283691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eef25a-54b6-4bb5-a863-6fe17ec58af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:34.284383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eef25a-54b6-4bb5-a863-6fe17ec58af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:34.294417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eef25a-54b6-4bb5-a863-6fe17ec58af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:34.296161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eef25a-54b6-4bb5-a863-6fe17ec58af2 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:34.313822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1071/2155] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:34.422800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef3edf8fa8844d2919c8faaa2e47804'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:34.432696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef3edf8fa8844d2919c8faaa2e47804'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:34.447316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef3edf8fa8844d2919c8faaa2e47804'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:34.448323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef3edf8fa8844d2919c8faaa2e47804'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:34.452989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:34.455473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1085/2156] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:33 2022] DELETE /identity/v3/users/aef3edf8fa8844d2919c8faaa2e47804 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:34.460244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:34.460907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:34.461389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:34.487957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:34.489182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:34.561392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1072/2157] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:34.577704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:34.579486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:34.649182 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265da6c-e62e-4bf1-96d3-bbb4e9b247a3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:34.653033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:34.653648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:34.653930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:34.654199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:34.663590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:34.665728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:34.678531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1086/2158] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:35.388888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:35.390518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:35.405841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356d49-6fb6-4121-8b9b-4297018b367b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:35.411690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:35.417029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:35.417029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:35.417029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] PATH_INFO: `/v3/users/fda00e4a39a1494ba8c083d51e106934` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:35.417029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda00e4a39a1494ba8c083d51e106934)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:35.417029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:35.516930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:35.521484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:35.549417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65ff0e-0c24-4045-bc48-c24121a458b6 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:35.557219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:35.558353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:35.559063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:35.560309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] PATH_INFO: `/v3/projects/696bce9f557e4ebd805e82474e3ecf44` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:35.564085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] RBAC: Authorizing `identity:delete_project(project_id=696bce9f557e4ebd805e82474e3ecf44)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:35.566069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:35.679879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda00e4a39a1494ba8c083d51e106934'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.680605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda00e4a39a1494ba8c083d51e106934'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.713891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda00e4a39a1494ba8c083d51e106934'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.714739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda00e4a39a1494ba8c083d51e106934'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.727561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0c3a0355-ac03-44ac-b526-4a246d47f1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda00e4a39a1494ba8c083d51e106934'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.778088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:35.789833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696bce9f557e4ebd805e82474e3ecf44', 'request_id': 'req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.833281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696bce9f557e4ebd805e82474e3ecf44', 'request_id': 'req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.847855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1073/2159] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:35 2022] DELETE /identity/v3/users/fda00e4a39a1494ba8c083d51e106934 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:35.852920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280f30-b6f5-4348-9e32-b526d68c5b59 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:35.853454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280f30-b6f5-4348-9e32-b526d68c5b59 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:35.853918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280f30-b6f5-4348-9e32-b526d68c5b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:35.893520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8b280f30-b6f5-4348-9e32-b526d68c5b59 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:35.896879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8b280f30-b6f5-4348-9e32-b526d68c5b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:35.948634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696bce9f557e4ebd805e82474e3ecf44', 'request_id': 'req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.966319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] The token cache is being invalidate because project 696bce9f557e4ebd805e82474e3ecf44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:35.966615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.968308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:35.971362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1087/2160] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:35 2022] DELETE /identity/v3/projects/696bce9f557e4ebd805e82474e3ecf44 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:36.004741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1074/2161] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:36.020438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:36.022924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:36.178010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa09a7b-941d-46d1-b59f-1d75b80dd06f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:36.278595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:36.280194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:36.280736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:36.281290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:36.298052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:36.298052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:36.312130 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1088/2162] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:36.634476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:36.635034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:36.635034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:36.695256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] MFA Rules not processed for user `0c308dbb461a48848ff8820889149c52`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:36.696744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:36.863480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1075/2163] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:36.870668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:36.873763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:36.898135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8619a-b3fa-4b48-a94a-8a9f0a2b81f8 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:36.941257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:36.942231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:36.942730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:36.943291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:36.944205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:36.946811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:36.985682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:36.987281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:37.040087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ffbeb-2747-4b99-bd2f-267dcb00cd82 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.049846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.051634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.052141 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.052565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] PATH_INFO: `/v3/projects/5f0382c560dd4a8cbd4cacc29e97823d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.054107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0382c560dd4a8cbd4cacc29e97823d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.056866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.119068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1089/2164] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:36 2022] POST /identity/v3/projects => generated 371 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:37.121791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.231562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0382c560dd4a8cbd4cacc29e97823d', 'request_id': 'req-84a55515-5222-4488-9fca-9d913e018504'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:37.244749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0382c560dd4a8cbd4cacc29e97823d', 'request_id': 'req-84a55515-5222-4488-9fca-9d913e018504'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:37.247025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd330-caed-4944-b90c-00926d0ba522 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.329411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.330447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.330912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.331512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.332398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.334813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.390501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0382c560dd4a8cbd4cacc29e97823d', 'request_id': 'req-84a55515-5222-4488-9fca-9d913e018504'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:37.404387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] The token cache is being invalidate because project 5f0382c560dd4a8cbd4cacc29e97823d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:37.405214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:37.407340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-84a55515-5222-4488-9fca-9d913e018504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:37.409325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1076/2165] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:36 2022] DELETE /identity/v3/projects/5f0382c560dd4a8cbd4cacc29e97823d => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:37.414456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c9d5-61f4-4a60-87ec-bf29dcbcc61e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.415064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c9d5-61f4-4a60-87ec-bf29dcbcc61e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.415452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224c9d5-61f4-4a60-87ec-bf29dcbcc61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.439546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b224c9d5-61f4-4a60-87ec-bf29dcbcc61e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:37.441253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b224c9d5-61f4-4a60-87ec-bf29dcbcc61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:37.451002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1077/2166] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:37.459231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.459855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.460443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.486636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:37.488395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:37.499574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1078/2167] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:37.507129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1090/2168] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:37 2022] POST /identity/v3/users => generated 292 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:37.509479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.511202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.513160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:37.527647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463f44c-0700-4d10-ab6e-64a4803fdee2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.530720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf05c0a-7f0b-465f-bdf8-b9e82de4a9e9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.538481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.538779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.539422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.541603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.541713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.541789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.541860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.541947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.542029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.542100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:37.542100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.543407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.543695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.558913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1091/2169] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:37.559104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1079/2170] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:37.561697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.561834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.572540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7188-0ee4-4cef-a814-b336183a9c8d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.572878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534dad6-d138-44a1-af01-2e1483f6c51f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.578389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.578589 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.579005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.579290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.579451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.579567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.579836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.580159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] PATH_INFO: `/v3/projects/697fde1b1aa447878c3e9ff1d004d194/users/ad9e136d0f34414d98bbfb4c11a87a4a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.580387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.582314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.607588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=697fde1b1aa447878c3e9ff1d004d194, user_id=ad9e136d0f34414d98bbfb4c11a87a4a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.614626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.673414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1080/2171] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:37 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:37.675863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.727346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1092/2172] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:37 2022] PUT /identity/v3/projects/697fde1b1aa447878c3e9ff1d004d194/users/ad9e136d0f34414d98bbfb4c11a87a4a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:37.731052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.746785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4c93d-12c7-4fb3-8af0-6484bdedf1b9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.831445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.832730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.833351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.833979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.835774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.839947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.843112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702af47-a841-472b-9e6c-b84ac2fe3b5b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.852176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.852821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.853101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.859947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.859947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.859947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.876557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1093/2173] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:37.881127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:37.906221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c74e4-993a-4d23-bf77-5d2a96c40bc2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:37.914146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:37.915316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:37.915819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:37.923010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] PATH_INFO: `/v3/projects/697fde1b1aa447878c3e9ff1d004d194/users/ad9e136d0f34414d98bbfb4c11a87a4a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:37.923010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697fde1b1aa447878c3e9ff1d004d194, user_id=ad9e136d0f34414d98bbfb4c11a87a4a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:37.925143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:37.938550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1094/2174] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:37 2022] PUT /identity/v3/projects/697fde1b1aa447878c3e9ff1d004d194/users/ad9e136d0f34414d98bbfb4c11a87a4a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:37.996126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:38.007802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1081/2175] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:37 2022] POST /identity/v3/users => generated 288 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:38.008009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:38.111720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707322-6203-4fdb-b962-4c584c18ae61 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:38.117999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:38.118997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:38.119506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:38.119996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:38.120899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:38.124492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:38.148103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1095/2176] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:38.149086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:38.166009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e5739-9ca2-42a1-8c39-101c162f79ac admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:38.173234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dc347-6f93-4af7-8908-fc2f5d2033c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:38.174336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dc347-6f93-4af7-8908-fc2f5d2033c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:38.174774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dc347-6f93-4af7-8908-fc2f5d2033c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:38.175236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dc347-6f93-4af7-8908-fc2f5d2033c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:38.176201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dc347-6f93-4af7-8908-fc2f5d2033c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:38.178263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dc347-6f93-4af7-8908-fc2f5d2033c2 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:38.200474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:38.207460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1082/2177] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:38.215336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6df0b-89ff-4e20-9f24-9530249dd0a3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:38.224515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:38.226954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:38.226954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:38.227558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] PATH_INFO: `/v3/projects/27e5db58b7ab4644962a1751f7090db0/users/e809ec3790ec4bfb967dfc8756845cfe/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:38.255657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e5db58b7ab4644962a1751f7090db0, user_id=e809ec3790ec4bfb967dfc8756845cfe, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:38.267406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:38.492679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1096/2178] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:38 2022] PUT /identity/v3/projects/27e5db58b7ab4644962a1751f7090db0/users/e809ec3790ec4bfb967dfc8756845cfe/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:38.500066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ecc1-785a-4072-b3b7-4536dca4e26b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:38.500360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ecc1-785a-4072-b3b7-4536dca4e26b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:38.500714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07ecc1-785a-4072-b3b7-4536dca4e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:38.561749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ea07ecc1-785a-4072-b3b7-4536dca4e26b None None] MFA Rules not processed for user `e809ec3790ec4bfb967dfc8756845cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:38.563265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ea07ecc1-785a-4072-b3b7-4536dca4e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:38.647552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1083/2179] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:38.659617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:38.665446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:38.747719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afb2ee-e139-4c07-9f3a-844bcda13aed admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:38.752697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:38.753368 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:38.753663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:38.753968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:38.768767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:38.770743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:38.787166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1097/2180] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:39.289825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:40.859236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:40.948197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dce9c-693d-4322-8b60-40d1155c43ff None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:40.948699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dce9c-693d-4322-8b60-40d1155c43ff None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:40.949208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63dce9c-693d-4322-8b60-40d1155c43ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:41.100369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c63dce9c-693d-4322-8b60-40d1155c43ff None None] MFA Rules not processed for user `ad9e136d0f34414d98bbfb4c11a87a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:41.102791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c63dce9c-693d-4322-8b60-40d1155c43ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:41.211047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1084/2181] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:41.228333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:41.240927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4bd7f-884c-4889-9a25-9e08898fdf0d service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:41.245635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:41.246513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:41.246966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:41.247443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:41.250109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:41.384218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:41.386113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:41.408684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1098/2182] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:41.665976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:41.666464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:41.666939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:41.735771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] MFA Rules not processed for user `2e02c6c1c3bc49e8a63f0533c3ea5391`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:41.737381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:41.835834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1085/2183] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:41.840704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:41.842156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:41.930643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585585a-b00d-48b3-8ff3-615501ccd831 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:41.935654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:41.936379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:41.936734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:41.937013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:41.938612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:41.940646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:42.081285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1099/2184] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:41 2022] POST /identity/v3/projects => generated 383 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:42.083007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:42.110586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d23b3f-11d4-4c10-b964-3e78a5a05a03 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:42.195632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:42.198165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:42.199237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:42.199520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:42.200628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:42.221943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:42.388524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1086/2185] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:42 2022] POST /identity/v3/users => generated 298 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:42.392429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:42.404746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e09448-0407-41d8-b78d-5efd17f31bf1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:42.412682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:42.414144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:42.414599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:42.415204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:42.415979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:42.417775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:42.434426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1100/2186] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:42.435924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:42.449511 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663137c7-ca0c-4f3e-bfa1-6ba7cd719802 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:42.456077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:42.457368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:42.457833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:42.458284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] PATH_INFO: `/v3/projects/01e2afef38524a88b5d9657aeed7fae9/users/5169a11ccba346b680684304feb98ff0/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:42.476640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e2afef38524a88b5d9657aeed7fae9, user_id=5169a11ccba346b680684304feb98ff0, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:42.479593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:42.533123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:42.581861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1087/2187] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:42 2022] PUT /identity/v3/projects/01e2afef38524a88b5d9657aeed7fae9/users/5169a11ccba346b680684304feb98ff0/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:42.583109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:42.594862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0af040-d9db-4c15-b0e0-06b1e7527609 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:42.666046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:42.666664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:42.666956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:42.667239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:42.667924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:42.669636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:42.684061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1101/2188] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:42.688515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:42.702056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bdb40-494c-4bf0-8125-4e818c99f620 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:42.709349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:42.713486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:42.713486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:42.713486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] PATH_INFO: `/v3/projects/01e2afef38524a88b5d9657aeed7fae9/users/5169a11ccba346b680684304feb98ff0/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:42.714924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e2afef38524a88b5d9657aeed7fae9, user_id=5169a11ccba346b680684304feb98ff0, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:42.717836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:42.732503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1088/2189] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:42 2022] PUT /identity/v3/projects/01e2afef38524a88b5d9657aeed7fae9/users/5169a11ccba346b680684304feb98ff0/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:28:42.813753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:43.367629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:43.439289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d03d5-71ca-46c9-bfc2-086571c8cf9f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:43.513483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49a4ec-a18c-4164-a3d9-5e0404c0cd5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:43.515313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49a4ec-a18c-4164-a3d9-5e0404c0cd5d service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:43.515694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49a4ec-a18c-4164-a3d9-5e0404c0cd5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:43.516365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49a4ec-a18c-4164-a3d9-5e0404c0cd5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:43.626302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49a4ec-a18c-4164-a3d9-5e0404c0cd5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:43.628464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49a4ec-a18c-4164-a3d9-5e0404c0cd5d service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:43.642984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1102/2190] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:43.905232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:44.023646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789eb59-7d6f-4e92-8df1-ca6851a118b8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:44.029356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:44.030016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:44.030439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:44.030718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] PATH_INFO: `/v3/users/e809ec3790ec4bfb967dfc8756845cfe` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:44.031965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e809ec3790ec4bfb967dfc8756845cfe)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:44.046560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:44.338060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e809ec3790ec4bfb967dfc8756845cfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:44.338060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e809ec3790ec4bfb967dfc8756845cfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:44.378755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e809ec3790ec4bfb967dfc8756845cfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:44.379630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e809ec3790ec4bfb967dfc8756845cfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:44.394096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e809ec3790ec4bfb967dfc8756845cfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:44.505492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1089/2191] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:43 2022] DELETE /identity/v3/users/e809ec3790ec4bfb967dfc8756845cfe => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:44.510135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:44.511224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:44.512124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:44.559377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:44.560686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:44.662043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1103/2192] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:44.690317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:44.694551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:44.799422 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91fc4c-fb28-424e-81d6-5c1ea3f1e9b8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:44.804384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7abd695-e0dd-40ff-a631-3c5726fadccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:44.806880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd695-e0dd-40ff-a631-3c5726fadccb service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:44.806880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd695-e0dd-40ff-a631-3c5726fadccb service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:44.806880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7abd695-e0dd-40ff-a631-3c5726fadccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:44.822573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abd695-e0dd-40ff-a631-3c5726fadccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:44.824285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7abd695-e0dd-40ff-a631-3c5726fadccb service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:44.836984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1090/2193] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:45.392375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:45.394085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:45.419064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a65fc7-9cf4-4cd7-b589-bc271bc2722b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:45.427758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:45.428507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:45.429651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:45.430211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] PATH_INFO: `/v3/projects/27e5db58b7ab4644962a1751f7090db0` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:45.439464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e5db58b7ab4644962a1751f7090db0)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:45.441737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:45.704521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e5db58b7ab4644962a1751f7090db0', 'request_id': 'req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:45.710375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e5db58b7ab4644962a1751f7090db0', 'request_id': 'req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:45.823549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e5db58b7ab4644962a1751f7090db0', 'request_id': 'req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:45.845336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] The token cache is being invalidate because project 27e5db58b7ab4644962a1751f7090db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:45.845850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:45.847184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf075c8e-0f27-4e8d-9bbb-ed312854e01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:45.851211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1104/2194] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:45 2022] DELETE /identity/v3/projects/27e5db58b7ab4644962a1751f7090db0 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:45.857175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:45.858040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:45.858040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:45.897869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:45.899756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:46.028947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:46.079937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1091/2195] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:46.087001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.088692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.088692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.202772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:46.205084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:46.221435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1105/2196] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:46.228329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:46.241007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8410e2-78e0-496d-a9aa-bbc5e5c3e878 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:46.253906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:46.255119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.255592 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.256085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.256986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:46.257489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:46.260067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:46.284124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1092/2197] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:46.291296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:46.311831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e57c1-f78e-4699-bd47-2d538bf3d319 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:46.318227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:46.319073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.319572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.320100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.320950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:46.323562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:46.499576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1106/2198] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:46 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:46.505461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:46.537785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b1c5f-edb7-4c8e-98fe-9da944b2edd6 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:46.646256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:46.668379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:46.669011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.669295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.671019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.671680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:46.673700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:46.867352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1093/2199] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:46 2022] POST /identity/v3/users => generated 287 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:46.871134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:46.883854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea63ee-84b5-4ae2-b959-bd76aa9ca6a4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:46.890029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:46.890741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.890930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.891457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.892309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:46.893944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:46.906408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1107/2200] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:46.910142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:46.922447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0938d-b738-44d2-8b92-8a3b33d21654 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:46.927243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb04ec-4681-41d4-85ce-e60a33ee4f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:46.927967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb04ec-4681-41d4-85ce-e60a33ee4f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.928334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb04ec-4681-41d4-85ce-e60a33ee4f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.928666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb04ec-4681-41d4-85ce-e60a33ee4f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.929382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb04ec-4681-41d4-85ce-e60a33ee4f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:46.931093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb04ec-4681-41d4-85ce-e60a33ee4f61 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:46.948848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1094/2201] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:46.954734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:46.968634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d9d7-119f-4e5d-a233-1ce9e28094da admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:46.975502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:46.976226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:46.976559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:46.976841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] PATH_INFO: `/v3/projects/49442d712d9b48f2a54854630a1eb437/users/eec6eba19216490e9b8ad39f47dc3e89/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:46.999301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49442d712d9b48f2a54854630a1eb437, user_id=eec6eba19216490e9b8ad39f47dc3e89, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:47.002263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:47.158345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1108/2202] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:46 2022] PUT /identity/v3/projects/49442d712d9b48f2a54854630a1eb437/users/eec6eba19216490e9b8ad39f47dc3e89/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:47.162966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:47.163191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:47.163428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:47.200204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] MFA Rules not processed for user `eec6eba19216490e9b8ad39f47dc3e89`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:47.201421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:47.271202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1095/2203] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:47.289234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:47.289234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:47.345789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4498efb-c67d-49e0-8862-2184ebb05749 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:47.349574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79993135-5c5f-4db8-be2e-13cf68bf3ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:47.350202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79993135-5c5f-4db8-be2e-13cf68bf3ffd service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:47.350479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79993135-5c5f-4db8-be2e-13cf68bf3ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:47.350756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79993135-5c5f-4db8-be2e-13cf68bf3ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:47.360372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79993135-5c5f-4db8-be2e-13cf68bf3ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:47.367006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79993135-5c5f-4db8-be2e-13cf68bf3ffd service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:47.371359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1109/2204] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:47.397180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:47.473579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f61f9-70ff-4432-af9e-332954d63597 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:47.479617 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:47.480383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:47.480666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:47.480944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] PATH_INFO: `/v3/users/eec6eba19216490e9b8ad39f47dc3e89` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:47.482216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec6eba19216490e9b8ad39f47dc3e89)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:47.485637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:47.984250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec6eba19216490e9b8ad39f47dc3e89'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:47.984996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec6eba19216490e9b8ad39f47dc3e89'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:47.998443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec6eba19216490e9b8ad39f47dc3e89'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:47.999176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec6eba19216490e9b8ad39f47dc3e89'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:48.006256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec6eba19216490e9b8ad39f47dc3e89'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:48.142887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1096/2205] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:28:47 2022] DELETE /identity/v3/users/eec6eba19216490e9b8ad39f47dc3e89 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:48.147374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:48.147849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:48.148070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:48.167947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:48.169172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:48.251451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1110/2206] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:48.262057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:48.265887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:48.395253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d1d46-9de6-43e5-96a2-ff3dfbbd5e8b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:48.400067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0ef71-c317-46c7-af79-affb63804236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:48.401625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0ef71-c317-46c7-af79-affb63804236 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:48.402012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0ef71-c317-46c7-af79-affb63804236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:48.402368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0ef71-c317-46c7-af79-affb63804236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:48.412975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0ef71-c317-46c7-af79-affb63804236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:48.414536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0ef71-c317-46c7-af79-affb63804236 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:48.425047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1097/2207] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:49.162670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:49.164771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:49.183424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c48a-3349-451c-956a-cfdf3f2b249c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:49.196425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:49.197236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:49.197521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:49.197825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] PATH_INFO: `/v3/projects/49442d712d9b48f2a54854630a1eb437` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:49.199048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49442d712d9b48f2a54854630a1eb437)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:49.201022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:49.423691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49442d712d9b48f2a54854630a1eb437', 'request_id': 'req-9f678495-bd64-4f7d-a2de-daa809b30150'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:49.438549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49442d712d9b48f2a54854630a1eb437', 'request_id': 'req-9f678495-bd64-4f7d-a2de-daa809b30150'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:49.604261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49442d712d9b48f2a54854630a1eb437', 'request_id': 'req-9f678495-bd64-4f7d-a2de-daa809b30150'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:49.623938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] The token cache is being invalidate because project 49442d712d9b48f2a54854630a1eb437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:28:49.624310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:49.630093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9f678495-bd64-4f7d-a2de-daa809b30150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:28:49.633544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1111/2208] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:28:49 2022] DELETE /identity/v3/projects/49442d712d9b48f2a54854630a1eb437 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:49.638311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:49.639924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:49.640376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:49.662211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:49.663417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:49.747042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1098/2209] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:49.748859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:49.749365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:49.749788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:49.788211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:49.789728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:49.800943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1112/2210] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:49.805686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:49.819183 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08418f3b-3dc0-465e-b241-fd980589854f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:49.828931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:49.829291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:49.829638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:49.830178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:49.831152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:28:49.831601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:49.833523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:49.868409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1099/2211] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:28:49.872567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:49.887947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436a49e-3d8e-4c5e-89ec-d9fb57e5b43f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:49.897029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:49.898183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:49.898907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:49.899359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:49.900208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:49.902668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:50.068827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1113/2212] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:28:49 2022] POST /identity/v3/projects => generated 389 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:28:50.069119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:50.085771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824eab-e14a-4eff-8c1d-747e9fd94a07 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:50.215753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:50.215753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:50.215753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:50.215753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:50.216214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:50.216214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:50.472631 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1100/2213] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:28:50 2022] POST /identity/v3/users => generated 295 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:28:50.476971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:50.489550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb02e3-ae56-472a-af71-3cb962152651 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:50.495373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:50.496345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:50.496712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:50.497096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:50.498060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:50.499903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:50.511594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1114/2214] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:28:50.516020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:50.528040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c498e-90f7-455a-b194-72ff78debe4d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:50.541753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2b750-2d27-4f7d-9fdc-d1e6f5f61d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:50.542388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2b750-2d27-4f7d-9fdc-d1e6f5f61d23 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:50.542726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2b750-2d27-4f7d-9fdc-d1e6f5f61d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:50.543104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2b750-2d27-4f7d-9fdc-d1e6f5f61d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:50.543706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2b750-2d27-4f7d-9fdc-d1e6f5f61d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:50.545842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2b750-2d27-4f7d-9fdc-d1e6f5f61d23 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:50.562733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1101/2215] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:28:50.563868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:50.576534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed38ab0-33eb-430b-a8c2-71cbbb2636f0 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:50.599079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:50.599766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:50.600095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:50.600371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] PATH_INFO: `/v3/projects/f33328e53bd9473da9f317bcb0fdc15b/users/91bbf21d41c4433a8f7ea522967a8244/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:50.618066 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33328e53bd9473da9f317bcb0fdc15b, user_id=91bbf21d41c4433a8f7ea522967a8244, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:50.620946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:50.750417 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1115/2216] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:28:50 2022] PUT /identity/v3/projects/f33328e53bd9473da9f317bcb0fdc15b/users/91bbf21d41c4433a8f7ea522967a8244/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:28:50.764669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb791310-7c0d-4296-bff3-7d44838cdec2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:50.764669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb791310-7c0d-4296-bff3-7d44838cdec2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:50.764669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb791310-7c0d-4296-bff3-7d44838cdec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:50.795797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-fb791310-7c0d-4296-bff3-7d44838cdec2 None None] MFA Rules not processed for user `91bbf21d41c4433a8f7ea522967a8244`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:50.797323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fb791310-7c0d-4296-bff3-7d44838cdec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:50.898206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1102/2217] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:50.908826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:50.910915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:51.003258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9e24a-e7c1-45fe-85e2-a25a1fa6ecf1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:51.008513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:51.009322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:51.009750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:51.010237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:51.021621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:51.024129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:51.036633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1116/2218] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:28:51.295574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894cbfb-7aee-4e86-9dd5-41d9bc41f494 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:51.295858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894cbfb-7aee-4e86-9dd5-41d9bc41f494 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:51.296168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894cbfb-7aee-4e86-9dd5-41d9bc41f494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:51.344788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f894cbfb-7aee-4e86-9dd5-41d9bc41f494 None None] MFA Rules not processed for user `5169a11ccba346b680684304feb98ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:51.348031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f894cbfb-7aee-4e86-9dd5-41d9bc41f494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:51.413254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:51.502036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1103/2219] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:28:51.515342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:51.522172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:51.605787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6873-c841-4448-bcaf-27ca42b772f9 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:51.609681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:51.610252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:51.610574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:51.612871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:51.612871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:51.723102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:51.724599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf363f05-a1ea-490a-b400-917913d244de service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:51.735009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1117/2220] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:28:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:28:51.860003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:51.860746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:51.863735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:51.885685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] MFA Rules not processed for user `b7e0a78238e94fa4a8209550c901a9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:51.887211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:51.947076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1104/2221] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:28:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:52.103445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:52.249779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74df0d-de4a-4a5f-87d7-1b982a3fe7f2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:52.252754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74df0d-de4a-4a5f-87d7-1b982a3fe7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:52.253243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74df0d-de4a-4a5f-87d7-1b982a3fe7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:52.305319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-3b74df0d-de4a-4a5f-87d7-1b982a3fe7f2 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:28:52.306844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3b74df0d-de4a-4a5f-87d7-1b982a3fe7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:52.421621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1118/2222] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:28:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:28:52.479426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:28:52.480974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:28:52.615107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76145c86-28b3-4f3a-bab3-ac396027652e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:28:52.621106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c8886-b19e-4a77-9465-f923f8c20982 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:28:52.621733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c8886-b19e-4a77-9465-f923f8c20982 service swift] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:28:52.622173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c8886-b19e-4a77-9465-f923f8c20982 service swift] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:28:52.622307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c8886-b19e-4a77-9465-f923f8c20982 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:28:52.632519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c8886-b19e-4a77-9465-f923f8c20982 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:28:52.634164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c8886-b19e-4a77-9465-f923f8c20982 service swift] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:28:52.663444 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1105/2223] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:28:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:28:54.277913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:55.048326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:28:58.465399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:00.474578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:01.921466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:02.360874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:03.229186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:04.528092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b09f0-d4c5-4d9e-b125-3dc800ee701f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:04.528722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b09f0-d4c5-4d9e-b125-3dc800ee701f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:04.528722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b09f0-d4c5-4d9e-b125-3dc800ee701f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:04.551955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-229b09f0-d4c5-4d9e-b125-3dc800ee701f None None] MFA Rules not processed for user `68c15654f5a14850a1763a020c17c849`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:04.554859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-229b09f0-d4c5-4d9e-b125-3dc800ee701f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:04.628063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1119/2224] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:05.583253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f8e14-f807-40d0-93c6-799d87d40ae1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:05.584240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f8e14-f807-40d0-93c6-799d87d40ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:05.584240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f8e14-f807-40d0-93c6-799d87d40ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:05.626392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-eb6f8e14-f807-40d0-93c6-799d87d40ae1 None None] MFA Rules not processed for user `2e02c6c1c3bc49e8a63f0533c3ea5391`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:05.626392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-eb6f8e14-f807-40d0-93c6-799d87d40ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:05.637937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1106/2225] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:29:06.590327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:08.137698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:08.691800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:09.829326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:10.571643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:10.955250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e35c4e-3558-4d0c-8557-50997992f201 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:10.956172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e35c4e-3558-4d0c-8557-50997992f201 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:10.956833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e35c4e-3558-4d0c-8557-50997992f201 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:10.959535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1120/2226] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:29:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:29:10.987950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:10.988237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:10.988516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:11.011698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] MFA Rules not processed for user `5c39ceb5a1fb48328b9ca050172da36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:11.013574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:11.098350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1107/2227] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:29:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:11.118697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e35c4e-3558-4d0c-8557-50997992f201 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:11.118697 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-15e35c4e-3558-4d0c-8557-50997992f201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:11.129334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e35c4e-3558-4d0c-8557-50997992f201 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:11.134909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91397f1f-d722-4679-9381-ceda1a3c1291 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:11.135591 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397f1f-d722-4679-9381-ceda1a3c1291 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:11.135871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397f1f-d722-4679-9381-ceda1a3c1291 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:11.136189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91397f1f-d722-4679-9381-ceda1a3c1291 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:11.146077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91397f1f-d722-4679-9381-ceda1a3c1291 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:11.147604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91397f1f-d722-4679-9381-ceda1a3c1291 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:11.158136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1121/2228] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:12.394897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:12.565101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:12.566935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:12.651117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d454-154f-48df-9554-16fedbca9637 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:12.651832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d454-154f-48df-9554-16fedbca9637 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:12.660410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d454-154f-48df-9554-16fedbca9637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:12.687439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e824d454-154f-48df-9554-16fedbca9637 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:12.691536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e824d454-154f-48df-9554-16fedbca9637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:12.703201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1122/2229] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:29:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:12.704302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5af44-f3dd-41af-9dea-9e84c50b57cf None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:12.709931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:12.710707 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:12.711085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:12.711457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] PATH_INFO: `/v3/users/0c308dbb461a48848ff8820889149c52` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:12.712954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c308dbb461a48848ff8820889149c52)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:12.715611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:12.720996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d454-154f-48df-9554-16fedbca9637 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:12.738918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d454-154f-48df-9554-16fedbca9637 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:12.746972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:12.748561 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:12.748990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:12.749277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:12.763976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:12.763976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:12.773961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1123/2230] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:29:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:29:13.046817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c308dbb461a48848ff8820889149c52'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:13.047790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c308dbb461a48848ff8820889149c52'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:13.065702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c308dbb461a48848ff8820889149c52'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:13.066641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c308dbb461a48848ff8820889149c52'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:13.078066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c308dbb461a48848ff8820889149c52'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:13.191004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1108/2231] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:29:12 2022] DELETE /identity/v3/users/0c308dbb461a48848ff8820889149c52 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:13.635916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:14.567605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:14.603458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410be7ac-e38d-4c6a-9593-fab1fac6ca06 service swift] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:14.818947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:14.820805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:14.821350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:14.823350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] PATH_INFO: `/v3/projects/0d02e1d5a80e4ee7b8dcd76f7024b147` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:14.826473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d02e1d5a80e4ee7b8dcd76f7024b147)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:14.829815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:15.032386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:15.137230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d02e1d5a80e4ee7b8dcd76f7024b147', 'request_id': 'req-db178d45-7adf-4490-9a95-272ca713c7a6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.164514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d02e1d5a80e4ee7b8dcd76f7024b147', 'request_id': 'req-db178d45-7adf-4490-9a95-272ca713c7a6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.246161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d02e1d5a80e4ee7b8dcd76f7024b147', 'request_id': 'req-db178d45-7adf-4490-9a95-272ca713c7a6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.285423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] The token cache is being invalidate because project 0d02e1d5a80e4ee7b8dcd76f7024b147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:29:15.286715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.287874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.300907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:15.301978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1124/2232] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:29:14 2022] DELETE /identity/v3/projects/0d02e1d5a80e4ee7b8dcd76f7024b147 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:15.314072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:15.330541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db70c0a-cd6c-44cd-9dc4-bcdb277bdf59 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:15.443849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:15.446029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:15.488912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:15.489813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:15.490195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:15.490569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] PATH_INFO: `/v3/users/ad9e136d0f34414d98bbfb4c11a87a4a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:15.492513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad9e136d0f34414d98bbfb4c11a87a4a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:15.494815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:15.597861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db178d45-7adf-4490-9a95-272ca713c7a6 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:15.631473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:15.635205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:15.643658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:15.644317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] PATH_INFO: `/v3/users/91bbf21d41c4433a8f7ea522967a8244` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:15.646874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91bbf21d41c4433a8f7ea522967a8244)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:15.651169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:15.804427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad9e136d0f34414d98bbfb4c11a87a4a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.808116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad9e136d0f34414d98bbfb4c11a87a4a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.848710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad9e136d0f34414d98bbfb4c11a87a4a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.850176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad9e136d0f34414d98bbfb4c11a87a4a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:15.863069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3d42aa13-2303-4006-ada8-ab4f8d21104b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad9e136d0f34414d98bbfb4c11a87a4a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:16.030090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91bbf21d41c4433a8f7ea522967a8244'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:16.070235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1109/2233] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:29:15 2022] DELETE /identity/v3/users/ad9e136d0f34414d98bbfb4c11a87a4a => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:29:16.114982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91bbf21d41c4433a8f7ea522967a8244'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:16.126367 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91bbf21d41c4433a8f7ea522967a8244'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:16.164071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91bbf21d41c4433a8f7ea522967a8244'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:16.165065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91bbf21d41c4433a8f7ea522967a8244'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:16.188536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1125/2234] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:29:15 2022] DELETE /identity/v3/users/91bbf21d41c4433a8f7ea522967a8244 => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:29:16.195285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:16.195285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:16.195285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:16.203443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:16.281949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:16.284969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:16.411056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1110/2235] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:16.449037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:16.449037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:16.554325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778f698-6dc3-40b7-bd1c-b9e4b4dac249 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:16.560842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39894aba-d703-4923-a64b-53d372159950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:16.562069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39894aba-d703-4923-a64b-53d372159950 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:16.562648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39894aba-d703-4923-a64b-53d372159950 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:16.563331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39894aba-d703-4923-a64b-53d372159950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:16.577655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39894aba-d703-4923-a64b-53d372159950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:16.579648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39894aba-d703-4923-a64b-53d372159950 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:16.594041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1126/2236] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:16.793674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:16.877071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:16.879644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:16.905837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4220-833c-4fff-a06f-6290c1b3618d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:16.936560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:16.937227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:16.937597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:16.937758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] PATH_INFO: `/v3/projects/697fde1b1aa447878c3e9ff1d004d194` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:16.941231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=697fde1b1aa447878c3e9ff1d004d194)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:16.944620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:17.133312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697fde1b1aa447878c3e9ff1d004d194', 'request_id': 'req-8b0933f7-794f-4c2e-a176-769731275800'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:17.148284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697fde1b1aa447878c3e9ff1d004d194', 'request_id': 'req-8b0933f7-794f-4c2e-a176-769731275800'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:17.228838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697fde1b1aa447878c3e9ff1d004d194', 'request_id': 'req-8b0933f7-794f-4c2e-a176-769731275800'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:17.245562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] The token cache is being invalidate because project 697fde1b1aa447878c3e9ff1d004d194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:29:17.245953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:17.246776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b0933f7-794f-4c2e-a176-769731275800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:17.250049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1111/2237] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:29:16 2022] DELETE /identity/v3/projects/697fde1b1aa447878c3e9ff1d004d194 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:29:17.257497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:17.257988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:17.262523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:17.301604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:17.306853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:17.481117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1127/2238] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:29:17.493353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:17.494287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:17.494287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:17.527739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:17.532020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:17.565207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1112/2239] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:17.571735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:17.589607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359426-618c-41c5-9352-288507bb7dd2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:17.599823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:17.601682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:17.602437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:17.603392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:17.604958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:29:17.605910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:17.609992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:17.626877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1128/2240] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:29:17.631683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:17.645526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85023-0201-40ea-a5fc-0a7d228d1c90 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:17.653317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:17.656650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:17.657325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:17.657798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:17.658834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:17.661229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:18.073126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:18.078048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:18.097976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23f4b-cee0-43af-baff-d79de63549ee admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:18.103903 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:18.104600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:18.105078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:18.105528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] PATH_INFO: `/v3/projects/f33328e53bd9473da9f317bcb0fdc15b` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:18.107570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33328e53bd9473da9f317bcb0fdc15b)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:18.110032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:18.166587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1113/2241] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:29:17 2022] POST /identity/v3/projects => generated 385 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:29:18.168613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:18.276382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47082454-3b31-4059-b8c8-40d1f34e00ff admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:18.287188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:18.287996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:18.288377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:18.288750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:18.290112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:18.292543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:18.322787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33328e53bd9473da9f317bcb0fdc15b', 'request_id': 'req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:18.332149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33328e53bd9473da9f317bcb0fdc15b', 'request_id': 'req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:18.492865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1114/2242] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:29:18 2022] POST /identity/v3/users => generated 293 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:29:18.497198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:18.509297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33328e53bd9473da9f317bcb0fdc15b', 'request_id': 'req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:18.511632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ec6e4-ea48-4fc6-b73a-896a935de7db admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:18.524805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] The token cache is being invalidate because project f33328e53bd9473da9f317bcb0fdc15b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:29:18.525404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:18.526364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:18.548393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1129/2243] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:29:18 2022] DELETE /identity/v3/projects/f33328e53bd9473da9f317bcb0fdc15b => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:29:18.600477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:18.600477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:18.600477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:18.601545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:18.601545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:18.603501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:18.619154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1115/2244] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:29:18.627686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:18.630157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:18.646586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d06aa2-5e76-48fc-9cee-9969b98ce0d8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:18.652878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0f72-cd75-44b0-b526-1785c932f438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:18.654007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0f72-cd75-44b0-b526-1785c932f438 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:18.654516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0f72-cd75-44b0-b526-1785c932f438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:18.654990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0f72-cd75-44b0-b526-1785c932f438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:18.656149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec0f72-cd75-44b0-b526-1785c932f438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:18.658489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec0f72-cd75-44b0-b526-1785c932f438 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:18.672749 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1130/2245] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:29:18.689223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:18.718382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b2969-8bf0-4227-bba0-01995f522207 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:18.731133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:18.732932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:18.733303 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:18.733654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] PATH_INFO: `/v3/projects/b3a4b63bdea640e18b2d452d59e06c2a/users/4a31a59e94e1453c9023b74aac8c808f/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:18.759989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3a4b63bdea640e18b2d452d59e06c2a, user_id=4a31a59e94e1453c9023b74aac8c808f, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:18.764983 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:18.917116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1116/2246] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:29:18 2022] PUT /identity/v3/projects/b3a4b63bdea640e18b2d452d59e06c2a/users/4a31a59e94e1453c9023b74aac8c808f/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:18.934624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:18.935654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:18.936172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:19.001314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] MFA Rules not processed for user `4a31a59e94e1453c9023b74aac8c808f`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:19.002620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:19.093228 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1131/2247] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:19.107413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:19.109477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:19.229147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9114989-bea6-4858-bf90-14658a973a46 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:19.233640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff93de-c5a1-456b-8409-fb0cb7011869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:19.234730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff93de-c5a1-456b-8409-fb0cb7011869 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:19.235231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff93de-c5a1-456b-8409-fb0cb7011869 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:19.235741 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff93de-c5a1-456b-8409-fb0cb7011869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:19.259419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff93de-c5a1-456b-8409-fb0cb7011869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:19.260919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff93de-c5a1-456b-8409-fb0cb7011869 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:19.271986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1117/2248] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:19.646247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:20.668051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:23.023396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:23.365728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:24.360575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:24.560596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:25.120004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:25.257151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:25.525865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:25.857811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:26.148775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:26.267863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:26.354408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:27.086184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:27.761770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:28.000375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:28.623335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:28.778380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:29.177406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:29.491460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:29.663389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:29.691399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:29.776929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:30.308852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:30.805911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:31.040623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:31.140253 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:31.237546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfac6ab-c634-4c79-ae2a-e513fbaedbf0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:31.242642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:31.243526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:31.244002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:31.244432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] PATH_INFO: `/v3/users/4a31a59e94e1453c9023b74aac8c808f` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:31.254763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a31a59e94e1453c9023b74aac8c808f)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:31.254763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:31.619234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a31a59e94e1453c9023b74aac8c808f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:31.765401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a31a59e94e1453c9023b74aac8c808f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:31.773184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a31a59e94e1453c9023b74aac8c808f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:31.787472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a31a59e94e1453c9023b74aac8c808f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:31.787472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a31a59e94e1453c9023b74aac8c808f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:31.792871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1132/2249] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:29:31 2022] DELETE /identity/v3/users/4a31a59e94e1453c9023b74aac8c808f => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:31.797535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:31.798024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:31.798118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:31.817951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:31.819085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:31.835758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:31.886383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1118/2250] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:31.896972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:31.898477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:31.935493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:31.961898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e42c8a-0fcc-4b8c-b421-fb041ed0cdde admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:31.966271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3addc8a-0eb4-4c45-910e-40598d473e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:31.966913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addc8a-0eb4-4c45-910e-40598d473e65 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:31.967251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addc8a-0eb4-4c45-910e-40598d473e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:31.967480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addc8a-0eb4-4c45-910e-40598d473e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:31.978278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3addc8a-0eb4-4c45-910e-40598d473e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:31.979809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3addc8a-0eb4-4c45-910e-40598d473e65 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:31.992143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1133/2251] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:32.297777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:32.590271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:32.592926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:32.616029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b4d34-31d3-4d4e-a49a-b97f989deabd None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:32.620649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:32.621294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:32.621575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:32.622030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] PATH_INFO: `/v3/projects/b3a4b63bdea640e18b2d452d59e06c2a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:32.623180 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3a4b63bdea640e18b2d452d59e06c2a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:32.625151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:32.697797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:32.765496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3a4b63bdea640e18b2d452d59e06c2a', 'request_id': 'req-7e114ec6-b65a-4a18-9452-835a310238da'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:32.773411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3a4b63bdea640e18b2d452d59e06c2a', 'request_id': 'req-7e114ec6-b65a-4a18-9452-835a310238da'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:32.902661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3a4b63bdea640e18b2d452d59e06c2a', 'request_id': 'req-7e114ec6-b65a-4a18-9452-835a310238da'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:32.918598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] The token cache is being invalidate because project b3a4b63bdea640e18b2d452d59e06c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:29:32.918892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:32.919748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-7e114ec6-b65a-4a18-9452-835a310238da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:32.922414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1119/2252] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:29:32 2022] DELETE /identity/v3/projects/b3a4b63bdea640e18b2d452d59e06c2a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:32.927812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:32.928428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:32.928599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:32.970020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:32.971363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:33.072956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1134/2253] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:33.143347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.143616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.143872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.167599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:33.169740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:33.178129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1120/2254] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:33.182765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:33.195299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d22643-aae8-4cef-b061-333aa5739757 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:33.201511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:33.202153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.202415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.202687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.203399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:29:33.203801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:33.205539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:33.220920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1135/2255] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:29:33.226753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:33.235496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:33.239961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b24d69-4dda-4c03-b26c-08e005b1dbab None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:33.244872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:33.245553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.245832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.246105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.246752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:33.249575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:33.392415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1121/2256] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:29:33 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:29:33.396667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:33.408973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62403ff7-9b56-4d2e-80d2-a7be9430c8d7 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:33.487656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:33.490239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.490239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.490239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.490239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:33.491563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:33.629982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1136/2257] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:29:33 2022] POST /identity/v3/users => generated 284 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:29:33.630175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:33.638781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b267bb9-f542-4e6d-b40d-66401952f2d9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:33.643354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:33.644000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.644282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.644555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.645195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:33.646799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:33.657719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1122/2258] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:29:33.662310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:33.663246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:33.673341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25cd4a-c817-4b35-8349-927175d3f4c0 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:33.679424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b5a52-fd7a-444a-a977-b38c939f638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:33.680266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b5a52-fd7a-444a-a977-b38c939f638f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.680553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b5a52-fd7a-444a-a977-b38c939f638f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.680830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b5a52-fd7a-444a-a977-b38c939f638f admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.681484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b5a52-fd7a-444a-a977-b38c939f638f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:33.683097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b5a52-fd7a-444a-a977-b38c939f638f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:33.697433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1137/2259] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:29:33.701384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:33.712147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201984-cd66-417c-b006-231aff7ea740 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:33.717917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:33.718679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.718980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.719255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] PATH_INFO: `/v3/projects/d5953d28a0e1415583bbf51bda6facb8/users/ec4feacab06a4bbdaec2586fbe2993dd/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.736102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5953d28a0e1415583bbf51bda6facb8, user_id=ec4feacab06a4bbdaec2586fbe2993dd, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:33.738796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:33.801045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:33.852962 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1123/2260] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:29:33 2022] PUT /identity/v3/projects/d5953d28a0e1415583bbf51bda6facb8/users/ec4feacab06a4bbdaec2586fbe2993dd/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:33.871704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:33.872022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:33.872301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:33.900611 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] MFA Rules not processed for user `ec4feacab06a4bbdaec2586fbe2993dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:33.901746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:33.929795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:33.970988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1138/2261] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:33.975866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.042574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bfa09-4e43-4722-a875-d54d8dfad16d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.048101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.048712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.053495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.053495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.053495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.053495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.079799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:34.149920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1124/2262] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:29:33 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:29:34.155024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.166631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa1814-9851-47c8-97a7-a52d6f5c70c9 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.227533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.228212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.228490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.228802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.229452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.231437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.332881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1139/2263] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:29:34 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:29:34.337363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.349469 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf31a2-f71e-41cf-9c3e-3d7473839ef8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.351189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:34.356021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.360060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.360060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.360060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.360060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.360060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.370407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1125/2264] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:29:34.374768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.385217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6a4c-dde7-42fc-a8e5-6f0c1be7f0a8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.390673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.391386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.391668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.391971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] PATH_INFO: `/v3/projects/306cab0f75764d898a3fb5b11c207e80/users/daf08917095a4a4f8c9f00fe02fe3a06/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.408921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=306cab0f75764d898a3fb5b11c207e80, user_id=daf08917095a4a4f8c9f00fe02fe3a06, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.411484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.522337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1140/2265] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:29:34 2022] PUT /identity/v3/projects/306cab0f75764d898a3fb5b11c207e80/users/daf08917095a4a4f8c9f00fe02fe3a06/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:34.526193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.538929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae379b-6626-4fc5-89c2-607bcc4b3712 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.608581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88644e-bcfd-4843-8578-ebbfae4eebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.609219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88644e-bcfd-4843-8578-ebbfae4eebd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.609634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88644e-bcfd-4843-8578-ebbfae4eebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.609936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88644e-bcfd-4843-8578-ebbfae4eebd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.610600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88644e-bcfd-4843-8578-ebbfae4eebd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.613998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88644e-bcfd-4843-8578-ebbfae4eebd9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.630043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1126/2266] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:29:34.630355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.648059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197422-f151-4848-afd5-091354214fb9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.653291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.654510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.655001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.655435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] PATH_INFO: `/v3/projects/306cab0f75764d898a3fb5b11c207e80/users/daf08917095a4a4f8c9f00fe02fe3a06/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.658558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=306cab0f75764d898a3fb5b11c207e80, user_id=daf08917095a4a4f8c9f00fe02fe3a06, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.667379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.685937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1141/2267] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:29:34 2022] PUT /identity/v3/projects/306cab0f75764d898a3fb5b11c207e80/users/daf08917095a4a4f8c9f00fe02fe3a06/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:29:34.689220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.689655 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.689880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.725850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] MFA Rules not processed for user `daf08917095a4a4f8c9f00fe02fe3a06`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:34.727086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:34.783552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:34.810958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1127/2268] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:29:34.821960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:34.823686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:34.895365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eac9bc-aa1b-47d1-9a6e-0ded266f5d77 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:34.901099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:34.903556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:34.903556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:34.903556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:34.951175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:34.977230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:34.978785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:34.990273 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1142/2269] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:38.480986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:38.551076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:38.553118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:38.558596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e116760c-5cdf-428c-9ccf-1e05c3b6835f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:38.565252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:38.566151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:38.566583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:38.569166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] PATH_INFO: `/v3/users/ec4feacab06a4bbdaec2586fbe2993dd` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:38.569166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4feacab06a4bbdaec2586fbe2993dd)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:38.570945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:38.597443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:38.681924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc8178-ce10-44c9-b3c8-2e8aedff75e4 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:38.688089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:38.689643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:38.689643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:38.689643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:38.691075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:38.705555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:38.708002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d9a72-dfd1-4750-87e8-717127b57920 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:38.729339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1143/2270] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:38.889429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4feacab06a4bbdaec2586fbe2993dd'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:38.890266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4feacab06a4bbdaec2586fbe2993dd'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:38.903446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4feacab06a4bbdaec2586fbe2993dd'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:38.904176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4feacab06a4bbdaec2586fbe2993dd'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:38.920120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4feacab06a4bbdaec2586fbe2993dd'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:39.006475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1128/2271] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:29:38 2022] DELETE /identity/v3/users/ec4feacab06a4bbdaec2586fbe2993dd => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:39.011096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:39.011459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:39.011842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:39.032874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:39.034154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:39.120595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1144/2272] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:29:39.137368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:39.139603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:39.239117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b490f-aa39-4a9f-883f-60c17be5ec87 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:39.243531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:39.244272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:39.244619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:39.244897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:39.257307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:39.259098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:39.273105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1129/2273] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:39.827838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:39.833617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:39.859126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c9d25-8816-4c1e-a864-b582074d8d1b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:39.864526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:39.865449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:39.866052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:39.866332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] PATH_INFO: `/v3/projects/d5953d28a0e1415583bbf51bda6facb8` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:39.869207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5953d28a0e1415583bbf51bda6facb8)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:39.869680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:40.080245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5953d28a0e1415583bbf51bda6facb8', 'request_id': 'req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.089629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5953d28a0e1415583bbf51bda6facb8', 'request_id': 'req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.093792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:40.228490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5953d28a0e1415583bbf51bda6facb8', 'request_id': 'req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.241963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] The token cache is being invalidate because project d5953d28a0e1415583bbf51bda6facb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:29:40.242323 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.242975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.244698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1145/2274] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:29:39 2022] DELETE /identity/v3/projects/d5953d28a0e1415583bbf51bda6facb8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:29:40.248894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:40.259292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2f097-e741-459c-946a-1dcfd10de5b8 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:40.323518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:40.324813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:40.324813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:40.325084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] PATH_INFO: `/v3/users/daf08917095a4a4f8c9f00fe02fe3a06` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:40.334552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf08917095a4a4f8c9f00fe02fe3a06)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:40.334552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:40.658082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:40.679633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf08917095a4a4f8c9f00fe02fe3a06'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.680640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf08917095a4a4f8c9f00fe02fe3a06'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.714781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf08917095a4a4f8c9f00fe02fe3a06'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.715754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf08917095a4a4f8c9f00fe02fe3a06'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.729400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf08917095a4a4f8c9f00fe02fe3a06'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:40.881152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1130/2275] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:29:40 2022] DELETE /identity/v3/users/daf08917095a4a4f8c9f00fe02fe3a06 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:41.654139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:41.657918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:41.675424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f1a2-fe4a-4938-99ff-a3ee8f04f767 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:41.747874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:41.748244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:41.748521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:41.748795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] PATH_INFO: `/v3/projects/306cab0f75764d898a3fb5b11c207e80` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:41.749971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=306cab0f75764d898a3fb5b11c207e80)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:41.751873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:42.000898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '306cab0f75764d898a3fb5b11c207e80', 'request_id': 'req-a508a7be-524c-4613-bff4-b3bebb2665e3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:42.007029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '306cab0f75764d898a3fb5b11c207e80', 'request_id': 'req-a508a7be-524c-4613-bff4-b3bebb2665e3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:42.092080 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '306cab0f75764d898a3fb5b11c207e80', 'request_id': 'req-a508a7be-524c-4613-bff4-b3bebb2665e3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:42.104013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] The token cache is being invalidate because project 306cab0f75764d898a3fb5b11c207e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:29:42.104419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:42.104957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a508a7be-524c-4613-bff4-b3bebb2665e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:29:42.106491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1146/2276] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:29:41 2022] DELETE /identity/v3/projects/306cab0f75764d898a3fb5b11c207e80 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:29:48.457458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:48.458905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:48.517165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f0b34-e24c-421c-9772-68ccd3ea6414 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:48.565896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:48.566682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:48.566801 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:48.567045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:48.568807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:48.632945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:48.634362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:48.644136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1131/2277] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:29:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:29:52.100851 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=55 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:57.815737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ddf4d-c7ab-44e7-9dae-e40c6862f498 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:57.816584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ddf4d-c7ab-44e7-9dae-e40c6862f498 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:57.816584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ddf4d-c7ab-44e7-9dae-e40c6862f498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:57.846867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b68ddf4d-c7ab-44e7-9dae-e40c6862f498 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:29:57.849091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b68ddf4d-c7ab-44e7-9dae-e40c6862f498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:57.914747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1147/2278] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:29:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:29:57.940041 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:29:57.941605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:29:57.963480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:29:58.016726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611f365-3824-4101-af7e-f066283dba64 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:29:58.022334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af1f3-001e-4c0f-bf7b-18322bb8e1f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:29:58.022950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af1f3-001e-4c0f-bf7b-18322bb8e1f3 service swift] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:29:58.023228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af1f3-001e-4c0f-bf7b-18322bb8e1f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:29:58.023522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af1f3-001e-4c0f-bf7b-18322bb8e1f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:29:58.033187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af1f3-001e-4c0f-bf7b-18322bb8e1f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:29:58.035207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af1f3-001e-4c0f-bf7b-18322bb8e1f3 service swift] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:29:58.044934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1132/2279] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:29:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:30:07.907827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=25 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:08.091915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=24 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:16.563075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd72215-2344-4fc3-8209-508384eb384f None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:16.564126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd72215-2344-4fc3-8209-508384eb384f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:16.564126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd72215-2344-4fc3-8209-508384eb384f None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:16.565187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1148/2280] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 21:30:16.573474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db25036-7b87-4422-9e64-4a834e7126df None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:16.575871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db25036-7b87-4422-9e64-4a834e7126df None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:16.575871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db25036-7b87-4422-9e64-4a834e7126df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:16.600848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2db25036-7b87-4422-9e64-4a834e7126df None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:16.603057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2db25036-7b87-4422-9e64-4a834e7126df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:16.709584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1133/2281] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:30:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:16.724466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd72215-2344-4fc3-8209-508384eb384f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:16.727194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ddd72215-2344-4fc3-8209-508384eb384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:16.805288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd72215-2344-4fc3-8209-508384eb384f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:16.810158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e23c1-c363-405f-b055-54a6a97378dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:16.810789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e23c1-c363-405f-b055-54a6a97378dc service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:16.811068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e23c1-c363-405f-b055-54a6a97378dc service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:16.811341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e23c1-c363-405f-b055-54a6a97378dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:16.825488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e23c1-c363-405f-b055-54a6a97378dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:16.827264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e23c1-c363-405f-b055-54a6a97378dc service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:16.840735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1149/2282] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:30:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:30:17.095186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e81d-01f4-44fc-9fdf-ded7da41a3fd None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:17.099309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e81d-01f4-44fc-9fdf-ded7da41a3fd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:17.099309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368e81d-01f4-44fc-9fdf-ded7da41a3fd None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:17.099309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1134/2283] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:30:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:30:17.104568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39dcf-4989-4dab-bc2b-d8acfe9a4c51 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:17.104983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39dcf-4989-4dab-bc2b-d8acfe9a4c51 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:17.105336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e39dcf-4989-4dab-bc2b-d8acfe9a4c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:17.129793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-55e39dcf-4989-4dab-bc2b-d8acfe9a4c51 None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:17.132007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-55e39dcf-4989-4dab-bc2b-d8acfe9a4c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:17.144105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1150/2284] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:30:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:17.158699 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e81d-01f4-44fc-9fdf-ded7da41a3fd None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:17.161314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6368e81d-01f4-44fc-9fdf-ded7da41a3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:17.186200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368e81d-01f4-44fc-9fdf-ded7da41a3fd None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:17.190749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee6961-d32c-461f-9369-300a71bce730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:17.191512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6961-d32c-461f-9369-300a71bce730 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:17.191794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6961-d32c-461f-9369-300a71bce730 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:17.192103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee6961-d32c-461f-9369-300a71bce730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:17.201647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee6961-d32c-461f-9369-300a71bce730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:17.203359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee6961-d32c-461f-9369-300a71bce730 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:17.213991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1135/2285] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:30:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:30:17.351302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e95b5-ed1e-4210-945a-008a242c2c27 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:17.351578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e95b5-ed1e-4210-945a-008a242c2c27 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:17.351816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e95b5-ed1e-4210-945a-008a242c2c27 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:17.353355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1151/2286] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:30:17.359721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb157fa-5610-4657-ae6d-61af69f81102 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:17.360148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb157fa-5610-4657-ae6d-61af69f81102 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:17.360436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb157fa-5610-4657-ae6d-61af69f81102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:17.383274 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7bb157fa-5610-4657-ae6d-61af69f81102 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:17.384581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7bb157fa-5610-4657-ae6d-61af69f81102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:17.395415 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1136/2287] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:30:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:30:17.407442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e95b5-ed1e-4210-945a-008a242c2c27 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:17.420550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e95b5-ed1e-4210-945a-008a242c2c27 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:17.424578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:17.425328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:17.425709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:17.426078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:17.439057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:17.440730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:17.456107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1152/2288] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:30:22.601586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532b41-acf2-41f8-936e-30d803858b28 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:22.602279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532b41-acf2-41f8-936e-30d803858b28 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:22.602279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532b41-acf2-41f8-936e-30d803858b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:22.624157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-01532b41-acf2-41f8-936e-30d803858b28 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:22.625499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-01532b41-acf2-41f8-936e-30d803858b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:22.633719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1137/2289] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:30:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:22.650909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:22.665058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8fa2f-0d68-4f76-84bc-50dafdf678bd service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:22.670242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:22.670876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:22.671149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:22.671423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:22.680854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:22.682487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:22.692255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1153/2290] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:30:32.617841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=43 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:32.688539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=41 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:48.494205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d4199-184e-460f-93ff-3d880a3d32c8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:48.494970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d4199-184e-460f-93ff-3d880a3d32c8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:48.500678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d4199-184e-460f-93ff-3d880a3d32c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:48.549687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ea3d4199-184e-460f-93ff-3d880a3d32c8 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:30:48.558696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ea3d4199-184e-460f-93ff-3d880a3d32c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:30:48.577361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1138/2291] 10.222.0.65 () {58 vars in 967 bytes} [Mon Aug 29 21:30:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:30:48.598658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:30:48.626941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506245-d8dd-45ed-9cf8-ed5de236bd43 service swift] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:30:48.639936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67de9c6-6699-49ab-9c1e-b2dcbb887b51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:30:48.641735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67de9c6-6699-49ab-9c1e-b2dcbb887b51 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:30:48.642272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67de9c6-6699-49ab-9c1e-b2dcbb887b51 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:30:48.642929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67de9c6-6699-49ab-9c1e-b2dcbb887b51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:30:48.656074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67de9c6-6699-49ab-9c1e-b2dcbb887b51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:30:48.658048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67de9c6-6699-49ab-9c1e-b2dcbb887b51 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:30:48.670768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1154/2292] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:30:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:30:58.541881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:30:58.666275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:31:01.747927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:01.748528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:01.748700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:01.770245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] MFA Rules not processed for user `68c15654f5a14850a1763a020c17c849`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:31:01.771476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:01.839328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1139/2293] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:31:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:31:11.832010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=25 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:31:15.756207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:15.756823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:15.757117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:15.790488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] MFA Rules not processed for user `b7e0a78238e94fa4a8209550c901a9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:31:15.792757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:15.893359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1155/2294] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:31:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:31:25.885645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=25 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:31:33.076749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:33.079129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:33.165230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafc40f-6d47-4a93-820f-f610c80792d6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:33.172835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:33.173487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:33.173779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:33.174081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:33.177085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:33.193851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:33.195486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a696e-92cc-4fb9-b34c-a5cec146588b service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:33.213443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1140/2295] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:31:36.796345 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:36.797923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:36.814211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268211b6-7c44-456b-affc-1e37fb96a7a0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:36.818179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:36.818802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:36.819085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:36.819357 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:36.821198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:36.836446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:36.838015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:36.852460 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1156/2296] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:31:43.203528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=26 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:31:46.846015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:31:50.923573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:50.924972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:50.925610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:50.950918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] MFA Rules not processed for user `d3c5edb153624912b87d61dd7ad5e4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:31:50.952243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:51.020496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1141/2297] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:31:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:31:51.038510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:51.040817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:51.076125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec44407-e0c1-4c49-9055-413f8c97fc1c service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:51.085338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:51.086387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:51.087116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:51.087814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:51.091810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:51.112953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:51.116561 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8464c-e5cf-44a9-a4e5-6d95e88f0c1c service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:51.144087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1157/2298] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:31:51.623461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:51.631338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:51.631794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:51.632311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:51.649937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5954a-d7ad-403e-8052-f8efc346e323 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:51.658392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:51.660327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:51.660327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:51.660698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:51.663760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:51.665944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:31:51.667509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:51.680268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1158/2299] 10.222.0.65 () {58 vars in 967 bytes} [Mon Aug 29 21:31:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:31:51.682622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:51.686121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6858f-3806-4002-891e-029e1ffaa451 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:51.707959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1142/2300] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:31:51.971779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:51.993297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471f892-abbb-4b4f-8951-6c9aab24ffd3 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:52.002022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:52.002710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:52.002995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:52.003268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:52.017565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:52.020098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:52.031964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1159/2301] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:31:52.140024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:52.140671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:52.141008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:52.171162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] MFA Rules not processed for user `0e3f8c861a9c49119fc531e2a159ec9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:31:52.173038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:52.238780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1143/2302] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:31:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:31:52.379461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:52.381716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:52.495456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4361c-f31e-46ab-9f9f-e843ab51efd9 service swift] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:52.502664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:52.503311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:52.503821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:52.504134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:52.506176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:52.519295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:52.521212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:52.535815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1160/2303] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:31:52.668302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:52.688113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6803b0f-a725-43b1-bd9d-fe29734a0eb9 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:52.694443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:52.695318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:52.696047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:52.696506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:52.700360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:52.724360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:52.724360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:52.728827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1144/2304] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:31:58.624656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:31:58.642291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd09826-6d99-4741-9f83-da0453b9d6a3 service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:31:58.649209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:31:58.650496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:31:58.651078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:31:58.651688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:31:58.655634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:31:58.677727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:31:58.680819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:31:58.700532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1161/2305] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:31:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:02.725344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=62 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:03.502241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:03.503665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:03.515464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09244b32-f103-484d-97c2-402509bdef9c service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:03.519561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:03.520155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:03.520437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:03.520712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:03.523470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:03.536069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:03.537430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:03.547913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1145/2306] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:32:05.303106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:05.337734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751ec72-e38e-4381-a506-41f3702802df service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:05.358857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:05.359657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:05.363470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:05.365389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:05.372665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:05.390848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:05.393520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:05.416190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1162/2307] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:05.763515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:05.764831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:05.853612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd461f15-54c6-4aa4-bedd-59cb3b54a19f service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:05.861001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:05.861964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:05.862384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:05.862789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] PATH_INFO: `/v3/users/2e02c6c1c3bc49e8a63f0533c3ea5391` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:05.864660 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e02c6c1c3bc49e8a63f0533c3ea5391)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:05.868173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:06.150876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e02c6c1c3bc49e8a63f0533c3ea5391'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:06.151693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e02c6c1c3bc49e8a63f0533c3ea5391'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:06.170361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e02c6c1c3bc49e8a63f0533c3ea5391'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:06.171562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e02c6c1c3bc49e8a63f0533c3ea5391'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:06.187082 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e02c6c1c3bc49e8a63f0533c3ea5391'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:06.284275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1146/2308] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:32:05 2022] DELETE /identity/v3/users/2e02c6c1c3bc49e8a63f0533c3ea5391 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:06.740905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:06.763065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0e27-2d1a-48cc-825a-81c1bd54567a service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:06.843920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:06.844599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:06.844878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:06.845151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] PATH_INFO: `/v3/projects/52ac0b73cab4494ab39d80f06c54e9da` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:06.846438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ac0b73cab4494ab39d80f06c54e9da)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:06.848745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:06.987911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:07.000735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ac0b73cab4494ab39d80f06c54e9da', 'request_id': 'req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.006464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ac0b73cab4494ab39d80f06c54e9da', 'request_id': 'req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.086431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ac0b73cab4494ab39d80f06c54e9da', 'request_id': 'req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.102339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] The token cache is being invalidate because project 52ac0b73cab4494ab39d80f06c54e9da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:32:07.102634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.103874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.105574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1163/2309] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:32:06 2022] DELETE /identity/v3/projects/52ac0b73cab4494ab39d80f06c54e9da => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:07.110446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:07.112568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:07.130455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9ed8c-19bd-416b-8205-909f48b36bfe admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:07.207738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:07.208470 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:07.209144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:07.209144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] PATH_INFO: `/v3/users/5169a11ccba346b680684304feb98ff0` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:07.210850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5169a11ccba346b680684304feb98ff0)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:07.212894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:07.222439 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:07.521995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5169a11ccba346b680684304feb98ff0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.523625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5169a11ccba346b680684304feb98ff0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.542394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5169a11ccba346b680684304feb98ff0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.543316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5169a11ccba346b680684304feb98ff0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.551930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5169a11ccba346b680684304feb98ff0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:07.794616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1147/2310] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:32:07 2022] DELETE /identity/v3/users/5169a11ccba346b680684304feb98ff0 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:08.097788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:08.100457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:08.116534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3accfeb-4018-4ab5-b218-a1d2a2ce527e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:08.225175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:08.225866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:08.226128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:08.226439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] PATH_INFO: `/v3/projects/01e2afef38524a88b5d9657aeed7fae9` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:08.227655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e2afef38524a88b5d9657aeed7fae9)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:08.229794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:08.395160 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e2afef38524a88b5d9657aeed7fae9', 'request_id': 'req-a99bee3f-eb48-40a2-8612-94db7b32fbf9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:08.401894 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e2afef38524a88b5d9657aeed7fae9', 'request_id': 'req-a99bee3f-eb48-40a2-8612-94db7b32fbf9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:08.525977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e2afef38524a88b5d9657aeed7fae9', 'request_id': 'req-a99bee3f-eb48-40a2-8612-94db7b32fbf9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:08.539531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] The token cache is being invalidate because project 01e2afef38524a88b5d9657aeed7fae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:32:08.539803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:08.540497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:08.542557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1164/2311] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:32:08 2022] DELETE /identity/v3/projects/01e2afef38524a88b5d9657aeed7fae9 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:12.074161 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:12.075808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:12.213147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5556a22-14ba-4077-83b4-a5886253cedf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:12.333796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:12.334683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:12.335205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:12.335646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:12.339474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:12.360371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:12.362166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:12.395639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1148/2312] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:32:14.524526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:14.526964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:14.560217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bee3f-eb48-40a2-8612-94db7b32fbf9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:14.574921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:14.577183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:14.578118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:14.579704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:14.583531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:14.624023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:14.627358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:14.650113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1165/2313] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:22.379461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:24.636714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=69,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:32.636543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:32.638053 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:32.709696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a261865-ddd4-48ad-8f62-5897e3a37152 service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:32.714468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:32.715104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:32.715515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:32.715833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:32.717791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:32.814420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:32.815988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452375e-b9a6-4253-beba-7621c39a0e12 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:32.827758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1149/2314] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:38.546472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:38.549277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:38.592122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46981f-665f-4bdf-aed0-05d6ea5ef032 service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:38.596309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:38.596889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:38.597336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:38.597738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] PATH_INFO: `/v3/users/b7e0a78238e94fa4a8209550c901a9a5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:38.600242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e0a78238e94fa4a8209550c901a9a5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:38.604973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:38.867399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e0a78238e94fa4a8209550c901a9a5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:38.901644 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:38.902643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:38.902643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:38.931816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] MFA Rules not processed for user `68c15654f5a14850a1763a020c17c849`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:38.934052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:39.008351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1150/2315] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:32:39.020810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e0a78238e94fa4a8209550c901a9a5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.035921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e0a78238e94fa4a8209550c901a9a5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.063349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e0a78238e94fa4a8209550c901a9a5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.068722 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d41f6773-1409-432a-aefd-88c9026b7ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e0a78238e94fa4a8209550c901a9a5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.076921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1166/2316] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:32:38 2022] DELETE /identity/v3/users/b7e0a78238e94fa4a8209550c901a9a5 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:39.441645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:39.469883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb92de-76c6-467b-93d2-e77e96ff673f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:39.584517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:39.600776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:39.602194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:39.602402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:39.602774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] PATH_INFO: `/v3/projects/bd7bb76694c2499895f24e611c644a27` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:39.605136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7bb76694c2499895f24e611c644a27)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:39.608533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:39.771138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7bb76694c2499895f24e611c644a27', 'request_id': 'req-5bcee86f-e0b3-4290-a3b6-a36278af1dda'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.781004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7bb76694c2499895f24e611c644a27', 'request_id': 'req-5bcee86f-e0b3-4290-a3b6-a36278af1dda'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.855950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7bb76694c2499895f24e611c644a27', 'request_id': 'req-5bcee86f-e0b3-4290-a3b6-a36278af1dda'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.880875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] The token cache is being invalidate because project bd7bb76694c2499895f24e611c644a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:32:39.881205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.883087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-5bcee86f-e0b3-4290-a3b6-a36278af1dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:39.891798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1151/2317] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:32:39 2022] DELETE /identity/v3/projects/bd7bb76694c2499895f24e611c644a27 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:39.896072 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:39.897606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:39.898004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:39.934247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:39.936215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:40.061297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1167/2318] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:40.075645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.076698 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.076900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.106816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:40.108149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:40.117984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1152/2319] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:32:40.125605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:40.142543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ad05e-7b8c-44ed-b931-3253593e1348 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:40.157965 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:40.159846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.159846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.159846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.160826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:32:40.161231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:40.163129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:40.191024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1168/2320] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:32:40.192903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:40.216786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12b866-e2ce-49ec-ae58-e15f2dd13127 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:40.227705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:40.228900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.229490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.231284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.233291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:40.236942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:40.455103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1153/2321] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:32:40 2022] POST /identity/v3/projects => generated 371 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:32:40.459459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:40.475559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc2a46-84c5-44cd-9c75-c3bfa082020d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:40.575560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:40.576634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.576634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.576867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.577988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:40.580496 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:40.593322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:40.784167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1169/2322] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:32:40 2022] POST /identity/v3/users => generated 286 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:32:40.790358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:40.805440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001727db-691a-4c8a-a041-4012e7754f16 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:40.811306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:40.811970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.812313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.812591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.813262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:40.814932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:40.838375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1154/2323] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:32:40.843439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:40.863401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e67a5-bb8b-4bd2-8fa4-86244e45c5c8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:40.872721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73420a2d-a5df-4fb8-8e8d-d280de9fb5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:40.873793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73420a2d-a5df-4fb8-8e8d-d280de9fb5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.874277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73420a2d-a5df-4fb8-8e8d-d280de9fb5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.874761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73420a2d-a5df-4fb8-8e8d-d280de9fb5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.875964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73420a2d-a5df-4fb8-8e8d-d280de9fb5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:40.879165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73420a2d-a5df-4fb8-8e8d-d280de9fb5b2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:40.902873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1170/2324] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:32:40.908943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:40.925384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeba31d-ee66-4c01-953e-a24696c3864e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:40.933195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:40.934690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:40.937942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:40.937942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] PATH_INFO: `/v3/projects/dbb0a73c46af4c3ea4258c0572d4ef7b/users/4e6e38ca5a2b4c4f8129e0b67fe032f4/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:40.960282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb0a73c46af4c3ea4258c0572d4ef7b, user_id=4e6e38ca5a2b4c4f8129e0b67fe032f4, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:40.966190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:41.169279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1155/2325] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:40 2022] PUT /identity/v3/projects/dbb0a73c46af4c3ea4258c0572d4ef7b/users/4e6e38ca5a2b4c4f8129e0b67fe032f4/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:41.183311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:41.183592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:41.183922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:41.221192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] MFA Rules not processed for user `4e6e38ca5a2b4c4f8129e0b67fe032f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:41.222481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:41.312441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1171/2326] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:41.326507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:41.328638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:41.464815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd992e3-02a0-4a3b-aaad-1900a038f714 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:41.475044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4f089-4742-4930-a904-345fb999a5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:41.479964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4f089-4742-4930-a904-345fb999a5e5 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:41.479964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4f089-4742-4930-a904-345fb999a5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:41.479964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4f089-4742-4930-a904-345fb999a5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:41.490758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4f089-4742-4930-a904-345fb999a5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:41.492488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4f089-4742-4930-a904-345fb999a5e5 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:41.504678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1156/2327] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:45.381865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:45.490991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb60e34-adb5-4459-b5f2-0f920764493e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:45.500725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:45.501844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:45.502309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:45.502717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] PATH_INFO: `/v3/users/4e6e38ca5a2b4c4f8129e0b67fe032f4` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:45.504685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6e38ca5a2b4c4f8129e0b67fe032f4)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:45.508493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:45.839289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6e38ca5a2b4c4f8129e0b67fe032f4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:45.942862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6e38ca5a2b4c4f8129e0b67fe032f4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:45.956621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6e38ca5a2b4c4f8129e0b67fe032f4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:45.971687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:45.983726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6e38ca5a2b4c4f8129e0b67fe032f4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:45.985531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6e38ca5a2b4c4f8129e0b67fe032f4'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:45.995523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1172/2328] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:32:45 2022] DELETE /identity/v3/users/4e6e38ca5a2b4c4f8129e0b67fe032f4 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:46.000473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:46.001011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:46.001226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:46.016348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:46.024352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:46.026413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:46.138775 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1157/2329] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:46.155531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:46.158354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:46.255680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ef32e-8032-449d-9967-390b514d3262 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:46.262682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438d24-bccb-4049-9804-e7a461b0da1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:46.263988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438d24-bccb-4049-9804-e7a461b0da1e service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:46.264401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438d24-bccb-4049-9804-e7a461b0da1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:46.264954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438d24-bccb-4049-9804-e7a461b0da1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:46.281631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438d24-bccb-4049-9804-e7a461b0da1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:46.283384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438d24-bccb-4049-9804-e7a461b0da1e service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:46.301654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1173/2330] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:47.281131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:47.283149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:47.301627 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c152d1c-a331-43af-b4db-0c8939ee2d8c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:47.307720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:47.308434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:47.308820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:47.309182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] PATH_INFO: `/v3/projects/dbb0a73c46af4c3ea4258c0572d4ef7b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:47.310739 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb0a73c46af4c3ea4258c0572d4ef7b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:47.313012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:47.449535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb0a73c46af4c3ea4258c0572d4ef7b', 'request_id': 'req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:47.457572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb0a73c46af4c3ea4258c0572d4ef7b', 'request_id': 'req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:47.559505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb0a73c46af4c3ea4258c0572d4ef7b', 'request_id': 'req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:47.580638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] The token cache is being invalidate because project dbb0a73c46af4c3ea4258c0572d4ef7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:32:47.581172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:47.582408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-a1d0c777-4416-4f18-9a9c-edbbdd4129bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:47.585340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1158/2331] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:32:47 2022] DELETE /identity/v3/projects/dbb0a73c46af4c3ea4258c0572d4ef7b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:47.592640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:47.593536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:47.593879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:47.621359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:47.622727 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:47.702516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1174/2332] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:47.716909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:47.717370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:47.717733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:47.753142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:47.755305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:47.770814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1159/2333] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:47.777892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:47.788724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380c92b-7e5e-4626-8594-0c046076189d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:47.794396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:47.795035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:47.795317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:47.795668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:47.796320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:32:47.796626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:47.798321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:47.816288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1175/2334] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:32:47.821060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:47.832300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a851b765-ffcb-4fa5-acc2-37ec7c4562c3 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:47.841361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:47.850232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:47.850232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:47.850232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:47.850232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:47.850232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:47.966506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1160/2335] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:32:47 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:32:47.972134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:47.984632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c150a-cb96-42ba-8672-58b395254297 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:48.055686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:48.056823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.057156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.057530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.058472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:48.060603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:48.175037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1176/2336] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:32:47 2022] POST /identity/v3/users => generated 282 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:32:48.182621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:48.196508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5b2c5-90c1-43a3-8e98-70ee71434b6f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:48.202556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:48.203217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.203492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.203769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.204482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:48.206135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:48.227701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1161/2337] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:32:48.234278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:48.253320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2367c9-ed0f-4c25-917c-92f9176de49c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:48.259626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c568c4-276d-49a8-a65c-dad94cbeb9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:48.260637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c568c4-276d-49a8-a65c-dad94cbeb9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.260931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c568c4-276d-49a8-a65c-dad94cbeb9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.261317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c568c4-276d-49a8-a65c-dad94cbeb9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.262128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c568c4-276d-49a8-a65c-dad94cbeb9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:48.264497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c568c4-276d-49a8-a65c-dad94cbeb9d3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:48.267073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:48.281351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1177/2338] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:32:48.286746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:48.298503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b214da80-444e-479a-aa36-99500a8faa54 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:48.304944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:48.306119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.307099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.307492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] PATH_INFO: `/v3/projects/7ee275f726c2429585cd571914c831cd/users/1c712db549024ff5975d785a4a0ff9f6/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.326256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee275f726c2429585cd571914c831cd, user_id=1c712db549024ff5975d785a4a0ff9f6, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:48.329207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:48.417058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1162/2339] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:48 2022] PUT /identity/v3/projects/7ee275f726c2429585cd571914c831cd/users/1c712db549024ff5975d785a4a0ff9f6/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:32:48.433912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.434284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.434681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.476570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] MFA Rules not processed for user `1c712db549024ff5975d785a4a0ff9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:48.478019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:48.547212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1178/2340] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:48.551996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:48.584289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:48.622795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962b497-562e-4e4e-ab3e-96ffe86d31dc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:48.630585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:48.631434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.631930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.632277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.637866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:48.637866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:48.752324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1163/2341] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:32:48 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:32:48.754136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:48.767642 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c99c0-2cfc-427d-b209-8ef67e56b565 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:48.860237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:48.860865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:48.861353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:48.861699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:48.862530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:48.865360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:48.999635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1179/2342] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:32:48 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:32:49.006098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:49.017090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc57b5-9f62-402a-a092-ac8ab1653a5b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:49.024379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:49.025486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:49.026187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:49.026673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:49.027936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:49.031102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:49.048759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1164/2343] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:32:49.056640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:49.068303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b11ee-a4cb-4d11-9db8-ce9725caeb9a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:49.073389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:49.074317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:49.074652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:49.074973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] PATH_INFO: `/v3/projects/4185d03afebf4100b67632a18bdde4b1/users/877dd687b75540d7ad031d716068490a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:49.093535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4185d03afebf4100b67632a18bdde4b1, user_id=877dd687b75540d7ad031d716068490a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:49.096814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:49.269350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1180/2344] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:49 2022] PUT /identity/v3/projects/4185d03afebf4100b67632a18bdde4b1/users/877dd687b75540d7ad031d716068490a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:49.273463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:49.286461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e26596-02e1-450e-96e8-2887a06c81da admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:49.346313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50149e43-4bec-43d5-83e2-5c7155dabc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:49.347019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50149e43-4bec-43d5-83e2-5c7155dabc17 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:49.347261 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50149e43-4bec-43d5-83e2-5c7155dabc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:49.347538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50149e43-4bec-43d5-83e2-5c7155dabc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:49.348246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50149e43-4bec-43d5-83e2-5c7155dabc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:49.349882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50149e43-4bec-43d5-83e2-5c7155dabc17 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:49.361256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1165/2345] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:32:49.365801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:49.376663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e8752-8178-4113-83a4-16275a91e94e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:49.381579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:49.382334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:49.382536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:49.382812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] PATH_INFO: `/v3/projects/4185d03afebf4100b67632a18bdde4b1/users/877dd687b75540d7ad031d716068490a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:49.384818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4185d03afebf4100b67632a18bdde4b1, user_id=877dd687b75540d7ad031d716068490a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:49.387502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:49.398475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1181/2346] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:49 2022] PUT /identity/v3/projects/4185d03afebf4100b67632a18bdde4b1/users/877dd687b75540d7ad031d716068490a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:49.411291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:49.411585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:49.411877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:49.443055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] MFA Rules not processed for user `877dd687b75540d7ad031d716068490a`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:49.444394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:49.531716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1166/2347] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:49.542923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:49.545138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:49.597013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:49.618676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa19dfa-be1c-495d-adda-9db16685c220 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:49.628375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ba241-6255-45f9-8743-a53abf33ab42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:49.628375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ba241-6255-45f9-8743-a53abf33ab42 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:49.628375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ba241-6255-45f9-8743-a53abf33ab42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:49.628375 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ba241-6255-45f9-8743-a53abf33ab42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:49.635599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ba241-6255-45f9-8743-a53abf33ab42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:49.637281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ba241-6255-45f9-8743-a53abf33ab42 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:49.648852 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1182/2348] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:50.896862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:50.925165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:50.973878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16bfce-26f8-4fe7-b51e-c19c8ed6791a None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:50.980718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:50.981380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:50.981657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:50.981930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] PATH_INFO: `/v3/users/1c712db549024ff5975d785a4a0ff9f6` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:50.983773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c712db549024ff5975d785a4a0ff9f6)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:50.985153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:51.319244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c712db549024ff5975d785a4a0ff9f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:51.319981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c712db549024ff5975d785a4a0ff9f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:51.333978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c712db549024ff5975d785a4a0ff9f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:51.334807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c712db549024ff5975d785a4a0ff9f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:51.345594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c712db549024ff5975d785a4a0ff9f6'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:51.494241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1167/2349] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:32:50 2022] DELETE /identity/v3/users/1c712db549024ff5975d785a4a0ff9f6 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:32:51.499736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:51.500572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:51.500572 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:51.532389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:51.534685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:51.613304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1183/2350] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:32:51.625095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:51.626825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:51.707132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d3089-b726-4c31-a3cd-b05a86fe0e12 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:51.711495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:51.713146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:51.713678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:51.714194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:51.731382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:51.734012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:51.750205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1168/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:32:52.351379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:52.353486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:52.370975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c304e27-582f-4a22-8998-ca7c65798bb4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:52.378099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:52.378099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:52.378551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:52.378717 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] PATH_INFO: `/v3/projects/7ee275f726c2429585cd571914c831cd` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:52.380054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee275f726c2429585cd571914c831cd)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:52.382351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:52.559534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee275f726c2429585cd571914c831cd', 'request_id': 'req-9079161e-000e-47d6-9df3-6f04c1c31ba7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:52.569401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee275f726c2429585cd571914c831cd', 'request_id': 'req-9079161e-000e-47d6-9df3-6f04c1c31ba7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:52.720648 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee275f726c2429585cd571914c831cd', 'request_id': 'req-9079161e-000e-47d6-9df3-6f04c1c31ba7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:52.735598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] The token cache is being invalidate because project 7ee275f726c2429585cd571914c831cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:32:52.736036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:52.736814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:52.738465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1184/2352] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:32:52 2022] DELETE /identity/v3/projects/7ee275f726c2429585cd571914c831cd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:52.743675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:52.755165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218808-cf7e-4956-8e3b-dfbe323cfa2e service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:52.830639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:52.856728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:52.857773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:52.858251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:52.858749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] PATH_INFO: `/v3/users/877dd687b75540d7ad031d716068490a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:52.860404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=877dd687b75540d7ad031d716068490a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:52.863981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:53.244874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '877dd687b75540d7ad031d716068490a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:53.245728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '877dd687b75540d7ad031d716068490a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:53.259657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '877dd687b75540d7ad031d716068490a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:53.260649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '877dd687b75540d7ad031d716068490a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:53.273016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '877dd687b75540d7ad031d716068490a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:53.472840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1169/2353] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:32:52 2022] DELETE /identity/v3/users/877dd687b75540d7ad031d716068490a => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:53.892120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:53.896973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:54.047036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:54.059567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079161e-000e-47d6-9df3-6f04c1c31ba7 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:54.127403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:54.134602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:54.208411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:54.209462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:54.213156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:54.213156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:54.213383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:54.226993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:54.230015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455a8e8-d83f-409b-9491-55abfa5acc19 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:54.244725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1185/2354] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:54.255372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1042d-82bd-4c23-9753-33e819a91722 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:54.263389 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:54.264377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:54.265535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:54.266331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] PATH_INFO: `/v3/projects/4185d03afebf4100b67632a18bdde4b1` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:54.268773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4185d03afebf4100b67632a18bdde4b1)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:54.270773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:54.432737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4185d03afebf4100b67632a18bdde4b1', 'request_id': 'req-0e889b25-ac97-4037-a88f-65a9052062df'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:54.445541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4185d03afebf4100b67632a18bdde4b1', 'request_id': 'req-0e889b25-ac97-4037-a88f-65a9052062df'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:54.562686 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4185d03afebf4100b67632a18bdde4b1', 'request_id': 'req-0e889b25-ac97-4037-a88f-65a9052062df'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:54.567729 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:54.577381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] The token cache is being invalidate because project 4185d03afebf4100b67632a18bdde4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:32:54.577695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:54.578416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e889b25-ac97-4037-a88f-65a9052062df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:32:54.579989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1170/2355] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:32:54 2022] DELETE /identity/v3/projects/4185d03afebf4100b67632a18bdde4b1 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:54.586165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:54.586550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:54.586929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:54.611701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:54.613093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:54.691198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1186/2356] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:32:54.702758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:54.703054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:54.703372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:54.730991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:54.739567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:54.752018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1171/2357] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:54.753414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:54.766809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cbd64-f802-40a1-9d43-8075458249fc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:54.773049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:54.774256 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:54.775069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:54.775363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:54.776214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:32:54.776518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:54.778171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:54.803310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1187/2358] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:32:54.810204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:54.824506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08653815-30a0-4bb4-b701-92ef7e6933b2 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:54.831057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:54.831863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:54.832531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:54.832851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:54.833815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:54.836746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:54.967380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1172/2359] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:32:54 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:32:54.973055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:54.991720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83b7b0-6914-478d-81f1-6ec4480788d8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:55.081011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:55.082039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:55.082562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:55.083081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:55.084348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:55.088899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:55.329857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1188/2360] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:32:54 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:32:55.335057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:55.347243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81368c5-d47a-400d-ba15-263300248531 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:55.352176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:55.352835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:55.353692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:55.353975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:55.354722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:55.356450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:55.367746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1173/2361] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:32:55.372679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:55.383707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8ef1e-34bb-407b-a1c3-75efc2869203 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:55.389480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16dc6c-26a5-4c4f-9b2e-fdf93cb9c8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:55.389927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16dc6c-26a5-4c4f-9b2e-fdf93cb9c8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:55.390204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16dc6c-26a5-4c4f-9b2e-fdf93cb9c8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:55.390475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16dc6c-26a5-4c4f-9b2e-fdf93cb9c8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:55.391127 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16dc6c-26a5-4c4f-9b2e-fdf93cb9c8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:55.392788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16dc6c-26a5-4c4f-9b2e-fdf93cb9c8a0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:55.405180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1189/2362] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:32:55.409503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:55.422204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4c351-0f97-42a1-9345-1da373abc0d3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:55.427590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:55.428483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:55.428856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:55.429191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] PATH_INFO: `/v3/projects/3b559a27cdbe446db161fa978b3620f5/users/59a7e0cec0de47449a962b8fdb8304fb/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:55.447814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b559a27cdbe446db161fa978b3620f5, user_id=59a7e0cec0de47449a962b8fdb8304fb, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:55.450913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:55.567814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1174/2363] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:55 2022] PUT /identity/v3/projects/3b559a27cdbe446db161fa978b3620f5/users/59a7e0cec0de47449a962b8fdb8304fb/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:55.579464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:55.579831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:55.581189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:55.613821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] MFA Rules not processed for user `59a7e0cec0de47449a962b8fdb8304fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:55.614978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:55.693758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1190/2364] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:55.698991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:55.785334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22e72e-2331-487e-b2fb-1f491c904821 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:55.791004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:55.791646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:55.791953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:55.792227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:55.792968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:55.795023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:55.937776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1175/2365] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:32:55 2022] POST /identity/v3/projects => generated 365 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:32:55.942747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:55.959679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b2c85-6482-4469-95f8-c7a479792a4a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:56.077955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:56.079048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.079569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.080094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.081340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:56.085425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:56.247237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1191/2366] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:32:55 2022] POST /identity/v3/users => generated 289 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:32:56.251437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:56.272355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbcc3f-9151-434d-9cd2-ba85214c7794 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:56.278524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:56.280078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.280078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.280078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.280760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:56.283151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:56.295258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1176/2367] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:32:56.301725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:56.325210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69c390-9f1f-4980-896c-d299dd9d2121 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:56.334297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:56.338153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.338153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.338153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] PATH_INFO: `/v3/projects/d7b4602351b742e48324128b9fc0ab94/users/cac596440de04b6f86b2e86a41539eee/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.373584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:56.377261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b4602351b742e48324128b9fc0ab94, user_id=cac596440de04b6f86b2e86a41539eee, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:56.381957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:56.533570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1192/2368] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:56 2022] PUT /identity/v3/projects/d7b4602351b742e48324128b9fc0ab94/users/cac596440de04b6f86b2e86a41539eee/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:32:56.538407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:56.549301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894e12c-bede-46e8-aa6e-a54bf4d47141 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:56.620287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ccdcf-c01e-4b01-b18d-b7f62b71a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:56.621231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccdcf-c01e-4b01-b18d-b7f62b71a95f admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.621580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccdcf-c01e-4b01-b18d-b7f62b71a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.621978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ccdcf-c01e-4b01-b18d-b7f62b71a95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.622743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ccdcf-c01e-4b01-b18d-b7f62b71a95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:56.624695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ccdcf-c01e-4b01-b18d-b7f62b71a95f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:56.637267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1177/2369] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:32:56.648574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:56.667440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ea728-2200-4480-be79-abbe6937f0c1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:56.674831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:56.675001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:56.675104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.675104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.675310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] PATH_INFO: `/v3/projects/d7b4602351b742e48324128b9fc0ab94/users/cac596440de04b6f86b2e86a41539eee/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.678362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b4602351b742e48324128b9fc0ab94, user_id=cac596440de04b6f86b2e86a41539eee, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:56.684200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:56.703509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1193/2370] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:32:56 2022] PUT /identity/v3/projects/d7b4602351b742e48324128b9fc0ab94/users/cac596440de04b6f86b2e86a41539eee/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:32:56.720258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.725537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.725537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.763932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] MFA Rules not processed for user `cac596440de04b6f86b2e86a41539eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:32:56.771272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:56.855337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1178/2371] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:32:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:32:56.873987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:56.876920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:56.975081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32adba1-73ce-41ae-a533-61a7f9c8d8dc admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:56.979977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:56.980709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:56.980993 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:56.981273 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:56.992742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:56.994690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:57.007150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1194/2372] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:32:57.286788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:32:57.429711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:32:57.437037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:32:57.457653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d782e0a-91a2-48e7-927c-be3823818207 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:32:57.468307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cd59-d829-47d3-ade9-da043efaed91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:32:57.470201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cd59-d829-47d3-ade9-da043efaed91 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:32:57.470201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cd59-d829-47d3-ade9-da043efaed91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:32:57.470201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cd59-d829-47d3-ade9-da043efaed91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:32:57.567523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0cd59-d829-47d3-ade9-da043efaed91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:32:57.569277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0cd59-d829-47d3-ade9-da043efaed91 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:32:57.583325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1179/2373] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:32:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:07.639483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=62 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:07.668456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=10 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:11.425431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:11.515627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8fdc-3a51-4b48-aa5d-ab55b709c8c6 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:11.520411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:11.521056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:11.521342 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:11.521620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] PATH_INFO: `/v3/users/59a7e0cec0de47449a962b8fdb8304fb` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:11.522794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a7e0cec0de47449a962b8fdb8304fb)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:11.524844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:11.827222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a7e0cec0de47449a962b8fdb8304fb'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:12.022221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a7e0cec0de47449a962b8fdb8304fb'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:12.031198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a7e0cec0de47449a962b8fdb8304fb'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:12.045212 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a7e0cec0de47449a962b8fdb8304fb'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:12.045963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a7e0cec0de47449a962b8fdb8304fb'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:12.051974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1195/2374] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:11 2022] DELETE /identity/v3/users/59a7e0cec0de47449a962b8fdb8304fb => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:12.056827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:12.064908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:12.064908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:12.078289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:12.079551 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:12.149545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1180/2375] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:12.161406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:12.162893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:12.227858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50cadc-1bb6-4059-9327-db2257841b65 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:12.232140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:12.232763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:12.233041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:12.233314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:12.242657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:12.244171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:12.254858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1196/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:12.834114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:12.836792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:12.851904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39348200-be0e-46fe-b077-452d9dc12050 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:12.859747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:12.860957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:12.861784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:12.861784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] PATH_INFO: `/v3/projects/3b559a27cdbe446db161fa978b3620f5` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:12.863391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b559a27cdbe446db161fa978b3620f5)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:12.866332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:13.084807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b559a27cdbe446db161fa978b3620f5', 'request_id': 'req-1fdde7ee-c98f-499f-9987-f5afd1b2f874'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.093394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b559a27cdbe446db161fa978b3620f5', 'request_id': 'req-1fdde7ee-c98f-499f-9987-f5afd1b2f874'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.295684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b559a27cdbe446db161fa978b3620f5', 'request_id': 'req-1fdde7ee-c98f-499f-9987-f5afd1b2f874'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.308942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] The token cache is being invalidate because project 3b559a27cdbe446db161fa978b3620f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:13.309419 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.310174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.312720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1181/2377] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:12 2022] DELETE /identity/v3/projects/3b559a27cdbe446db161fa978b3620f5 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:13.319915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:13.328759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb0b2-9bbc-40e3-b34e-dae27e39852a service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:13.359810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:13.410710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:13.411388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:13.411672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:13.411987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] PATH_INFO: `/v3/users/cac596440de04b6f86b2e86a41539eee` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:13.413362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac596440de04b6f86b2e86a41539eee)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:13.418085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:13.716786 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac596440de04b6f86b2e86a41539eee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.795662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac596440de04b6f86b2e86a41539eee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.803494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac596440de04b6f86b2e86a41539eee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.817668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac596440de04b6f86b2e86a41539eee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.818416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac596440de04b6f86b2e86a41539eee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:13.825593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1197/2378] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:13 2022] DELETE /identity/v3/users/cac596440de04b6f86b2e86a41539eee => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:14.426718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:14.428766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:14.443790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdde7ee-c98f-499f-9987-f5afd1b2f874 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:14.516580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:14.517237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:14.517531 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:14.517777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] PATH_INFO: `/v3/projects/d7b4602351b742e48324128b9fc0ab94` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:14.518957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b4602351b742e48324128b9fc0ab94)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:14.520973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:14.674642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:14.701295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b4602351b742e48324128b9fc0ab94', 'request_id': 'req-aba3d821-28a3-44db-b43e-3293e27df7b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:14.708132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b4602351b742e48324128b9fc0ab94', 'request_id': 'req-aba3d821-28a3-44db-b43e-3293e27df7b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:14.735109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:14.736599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:14.797300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d8c8b-1e0d-4d49-8086-21309adc5a45 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:14.845527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b4602351b742e48324128b9fc0ab94', 'request_id': 'req-aba3d821-28a3-44db-b43e-3293e27df7b7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:14.854097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:14.854097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:14.854560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:14.854946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:14.857653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:14.861043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] The token cache is being invalidate because project d7b4602351b742e48324128b9fc0ab94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:14.861343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:14.862029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-aba3d821-28a3-44db-b43e-3293e27df7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:14.863793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1182/2379] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:14 2022] DELETE /identity/v3/projects/d7b4602351b742e48324128b9fc0ab94 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:14.868912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:14.869509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:14.869745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:14.873020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:14.892335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:14.893665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:14.935288 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:14.936737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf32c7-74f5-47ae-bc15-6b669f8a75e8 service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:14.947951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1198/2380] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:33:14.965072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1183/2381] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:14.977322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:14.977751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:14.978216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.002474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:15.004195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:15.013451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1199/2382] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:15.017820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.028065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078ad66-8bf6-41bc-81f8-8bf6f3c1d29d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:15.035643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:15.036310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.040917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.040917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.040917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:15.040917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:15.040917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:15.056772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1184/2383] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:15.061802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.072663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85d7c-b561-4720-935e-4eb81c1ab2b3 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:15.077982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:15.079629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.079629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.079629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.080204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:15.081926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:15.190917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1200/2384] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:15 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:15.195044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.204856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3af09-e0fb-4309-9c03-2f1f2ac24a57 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:15.263485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:15.264123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.264403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.264671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.265312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:15.267286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:15.404716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1185/2385] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:15 2022] POST /identity/v3/users => generated 284 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:15.405669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.417505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf8d70-d60f-4cf5-b7dc-f955a4ba3099 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:15.422353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a32881-d25f-4789-a215-7a8b82011869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:15.422966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32881-d25f-4789-a215-7a8b82011869 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.423242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32881-d25f-4789-a215-7a8b82011869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.423513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32881-d25f-4789-a215-7a8b82011869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.424199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a32881-d25f-4789-a215-7a8b82011869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:15.426022 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a32881-d25f-4789-a215-7a8b82011869 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:15.446881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1201/2386] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:15.450382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.465349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31085b0f-25e1-4bbc-bfb0-3ce58c32076e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:15.473842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:15.476222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.476706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.477400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.478658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:15.481906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:15.500603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1186/2387] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:33:15.502679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628eb0fe-4706-4b0c-a005-133add2bf8f0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.502679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628eb0fe-4706-4b0c-a005-133add2bf8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.502679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628eb0fe-4706-4b0c-a005-133add2bf8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.506048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.526977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80fe3e-8e8f-48fb-9c79-d0616ced4137 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:15.527434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-628eb0fe-4706-4b0c-a005-133add2bf8f0 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:15.528778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-628eb0fe-4706-4b0c-a005-133add2bf8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:15.532295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:15.532985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.533530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.533813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] PATH_INFO: `/v3/projects/a58aa037dded4ce5bc5d58560c853564/users/2499647f60be41b9b7aff896c8a58d7e/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.558376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58aa037dded4ce5bc5d58560c853564, user_id=2499647f60be41b9b7aff896c8a58d7e, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:15.562603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:15.569191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:15.604915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1202/2388] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:15.661691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1187/2389] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:15 2022] PUT /identity/v3/projects/a58aa037dded4ce5bc5d58560c853564/users/2499647f60be41b9b7aff896c8a58d7e/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:15.676818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:15.677413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:15.677848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:15.742167 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] MFA Rules not processed for user `2499647f60be41b9b7aff896c8a58d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:15.746079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:15.828586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.830492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:15.883015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1203/2390] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:15.890828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:15.909088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:15.983877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1cdb4-97f7-4b8e-9ef7-aa395393540f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.007447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.010193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.011067 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.012185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.016804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cef3-7e87-4cc2-b512-89b357e0c30e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.021813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.022625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.023211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.029289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.029289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.029289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.082028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.083635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.093190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1188/2391] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:33:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:33:16.160500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1204/2392] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:15 2022] POST /identity/v3/projects => generated 369 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:16.162170 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:16.186649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe553a-cc06-494b-bf89-539fc7840da5 service swift] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.286430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.287272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.287678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.288103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.288907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.296453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.440547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1189/2393] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:16 2022] POST /identity/v3/users => generated 291 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:16.455440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:16.500924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b50af-9b11-4b0f-aa09-f4bce6b8bcd5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.514193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.516221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.516621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.516986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.517898 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.520847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.540084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1205/2394] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:16.543189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:16.557692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17850043-3d35-45e3-9ba9-7b649282a039 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.562806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.566127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.566127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.566127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] PATH_INFO: `/v3/projects/d56bd3cc126b4df9a91d51c5d0518920/users/23fb71324ebc472aa18a732da3225f61/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.604790 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56bd3cc126b4df9a91d51c5d0518920, user_id=23fb71324ebc472aa18a732da3225f61, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.607848 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.716051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1190/2395] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:16 2022] PUT /identity/v3/projects/d56bd3cc126b4df9a91d51c5d0518920/users/23fb71324ebc472aa18a732da3225f61/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:16.717445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:16.735083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18d01d-7ac0-4951-82c6-d590ee559d5a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.874416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f94c-7d77-460d-b247-3b7dfc01382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.875463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f94c-7d77-460d-b247-3b7dfc01382c admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.876056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f94c-7d77-460d-b247-3b7dfc01382c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.876531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f94c-7d77-460d-b247-3b7dfc01382c admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.878521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93f94c-7d77-460d-b247-3b7dfc01382c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.881797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93f94c-7d77-460d-b247-3b7dfc01382c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.906055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:16.907201 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1206/2396] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:16.914579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:16.929035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee0ea0-a54e-4eb2-9aa2-a487cb3a37cb admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:16.942483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:16.943550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:16.943918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:16.944173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] PATH_INFO: `/v3/projects/d56bd3cc126b4df9a91d51c5d0518920/users/23fb71324ebc472aa18a732da3225f61/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:16.948885 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56bd3cc126b4df9a91d51c5d0518920, user_id=23fb71324ebc472aa18a732da3225f61, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:16.961314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:16.997996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1191/2397] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:16 2022] PUT /identity/v3/projects/d56bd3cc126b4df9a91d51c5d0518920/users/23fb71324ebc472aa18a732da3225f61/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:17.009147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:17.009147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:17.009961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:17.096126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] MFA Rules not processed for user `23fb71324ebc472aa18a732da3225f61`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:17.100742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:17.245897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1207/2398] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:17.255816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:17.257710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:17.341320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0c560-0a83-4646-ae8b-53fb51613bdf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:17.358219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:17.358219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:17.358219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:17.358219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:17.368890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:17.371181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:17.383593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1192/2399] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:33:17.548891 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:17.550597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:17.558132 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:17.566261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb13ea5-443b-49ca-9997-67900881f219 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:17.570599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568854c-ebb0-4508-8491-e10db41dd38a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:17.571250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568854c-ebb0-4508-8491-e10db41dd38a service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:17.571540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568854c-ebb0-4508-8491-e10db41dd38a service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:17.571826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568854c-ebb0-4508-8491-e10db41dd38a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:17.674104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568854c-ebb0-4508-8491-e10db41dd38a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:17.676416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568854c-ebb0-4508-8491-e10db41dd38a service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:17.710688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1208/2400] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:24.241256 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:24.330489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2d049-2f4d-4070-a73f-b0d8685be0d0 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:24.335750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:24.336621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:24.336907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:24.337236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] PATH_INFO: `/v3/users/2499647f60be41b9b7aff896c8a58d7e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:24.338508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2499647f60be41b9b7aff896c8a58d7e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:24.340526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:24.685678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2499647f60be41b9b7aff896c8a58d7e'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:24.689290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2499647f60be41b9b7aff896c8a58d7e'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:24.710325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2499647f60be41b9b7aff896c8a58d7e'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:24.711308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2499647f60be41b9b7aff896c8a58d7e'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:24.721374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2499647f60be41b9b7aff896c8a58d7e'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:24.842643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1193/2401] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:24 2022] DELETE /identity/v3/users/2499647f60be41b9b7aff896c8a58d7e => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:24.847439 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:24.848043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:24.848043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:24.871764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:24.873584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:25.016713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1209/2402] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:25.027806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:25.029339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:25.091042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:25.191054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173ec9b-ab61-4bd7-92ea-704642b00ec8 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:25.198516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:25.199872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:25.200287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:25.200716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:25.212140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:25.214007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:25.229630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1194/2403] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:26.002986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:26.008935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:26.045541 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744f467-e15a-4a29-ba3c-93d7d0675f1f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:26.058369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:26.060015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:26.060315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:26.060596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] PATH_INFO: `/v3/projects/a58aa037dded4ce5bc5d58560c853564` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:26.066397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58aa037dded4ce5bc5d58560c853564)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:26.068757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:26.268086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58aa037dded4ce5bc5d58560c853564', 'request_id': 'req-4752b2de-2cd1-4801-b016-68bcee4a9709'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.275695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58aa037dded4ce5bc5d58560c853564', 'request_id': 'req-4752b2de-2cd1-4801-b016-68bcee4a9709'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.357268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58aa037dded4ce5bc5d58560c853564', 'request_id': 'req-4752b2de-2cd1-4801-b016-68bcee4a9709'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.381232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] The token cache is being invalidate because project a58aa037dded4ce5bc5d58560c853564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:26.381580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.382947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.384795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1210/2404] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:26 2022] DELETE /identity/v3/projects/a58aa037dded4ce5bc5d58560c853564 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:26.388808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:26.404077 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379eb488-67ed-4ddb-9103-26bfa027a624 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:26.504152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:26.504866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:26.505156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:26.505909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] PATH_INFO: `/v3/users/23fb71324ebc472aa18a732da3225f61` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:26.507541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23fb71324ebc472aa18a732da3225f61)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:26.509835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:26.807863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23fb71324ebc472aa18a732da3225f61'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.808589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23fb71324ebc472aa18a732da3225f61'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.829023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23fb71324ebc472aa18a732da3225f61'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.829910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23fb71324ebc472aa18a732da3225f61'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:26.842131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-de4d1cbb-0b18-4c0a-a299-77a9067662d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23fb71324ebc472aa18a732da3225f61'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:27.035021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1195/2405] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:26 2022] DELETE /identity/v3/users/23fb71324ebc472aa18a732da3225f61 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:28.195344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:28.197025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:28.227718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b2de-2cd1-4801-b016-68bcee4a9709 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:28.254092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:28.302332 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:28.304150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:28.304711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:28.305185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] PATH_INFO: `/v3/projects/d56bd3cc126b4df9a91d51c5d0518920` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:28.306932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56bd3cc126b4df9a91d51c5d0518920)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:28.309815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:28.508690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56bd3cc126b4df9a91d51c5d0518920', 'request_id': 'req-7996de83-ca2b-4f88-92a0-b19925a0125c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:28.522615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56bd3cc126b4df9a91d51c5d0518920', 'request_id': 'req-7996de83-ca2b-4f88-92a0-b19925a0125c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:28.650847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56bd3cc126b4df9a91d51c5d0518920', 'request_id': 'req-7996de83-ca2b-4f88-92a0-b19925a0125c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:28.668065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] The token cache is being invalidate because project d56bd3cc126b4df9a91d51c5d0518920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:28.668753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:28.669632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-7996de83-ca2b-4f88-92a0-b19925a0125c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:28.671916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1211/2406] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:28 2022] DELETE /identity/v3/projects/d56bd3cc126b4df9a91d51c5d0518920 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:28.678626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:28.678626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:28.678626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:28.707290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:28.708874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:28.787930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:28.838254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1196/2407] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:28.846839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:28.847184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:28.847483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:28.874198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:28.875426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:28.883677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1212/2408] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:28.887935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:28.897749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f512b80-550e-4d97-a52f-ecb764be2a31 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:28.903254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:28.906629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:28.906629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:28.906629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:28.906629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:28.906629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:28.907186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:28.920400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1197/2409] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:28.925475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:28.935911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66864022-6bd7-4074-92f6-fb479f34a534 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:28.941192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:28.947707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:28.947707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:28.947707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:28.947707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:28.947707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:29.134957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1213/2410] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:28 2022] POST /identity/v3/projects => generated 367 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:29.140434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:29.154947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bdbbcb-1686-4ea7-9d36-6ec7711c2660 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:29.223803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:29.224455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.224735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.225033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.225672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:29.227842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:29.348288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1198/2411] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:29 2022] POST /identity/v3/users => generated 284 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:29.353619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:29.366769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6f55b-2208-4551-8ada-beb0c53d0ec3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:29.372449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:29.372991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.373272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.373545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.374255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:29.375859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:29.388369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1214/2412] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:29.392094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:29.411669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92bd43-2754-4ff7-99fc-b9e434b46a12 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:29.418594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa9ee8-223f-485b-96c1-342aa55fc757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:29.419380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9ee8-223f-485b-96c1-342aa55fc757 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.419717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9ee8-223f-485b-96c1-342aa55fc757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.420036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa9ee8-223f-485b-96c1-342aa55fc757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.421133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9ee8-223f-485b-96c1-342aa55fc757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:29.422800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa9ee8-223f-485b-96c1-342aa55fc757 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:29.435844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1199/2413] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:29.440162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:29.452285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43242e-f9f2-4ff6-ace4-c608b4a000bc admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:29.465287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:29.466225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.466588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.466869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] PATH_INFO: `/v3/projects/64da9604bdf84d9cace5cf0c3c7bd067/users/cb355da653e04ec6ad48e8e66baf2f3f/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.485673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=64da9604bdf84d9cace5cf0c3c7bd067, user_id=cb355da653e04ec6ad48e8e66baf2f3f, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:29.488521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:29.565481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1215/2414] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:29 2022] PUT /identity/v3/projects/64da9604bdf84d9cace5cf0c3c7bd067/users/cb355da653e04ec6ad48e8e66baf2f3f/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:29.577641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.577959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.578266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.608720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] MFA Rules not processed for user `cb355da653e04ec6ad48e8e66baf2f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:29.610024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:29.677565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1200/2415] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:29.682258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:29.748231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:29.759579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18edb-4100-473a-8838-3530b496c9fd admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:29.765047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:29.765657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.765940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.766339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.767026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:29.769141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:29.880733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1216/2416] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:29 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:29.885021 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:29.898756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d410d2-64c7-4f0e-b7d5-626c33254605 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:29.927757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:29.965792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:29.967282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:29.967640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:29.968425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:29.969414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:29.971677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:30.089427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1201/2417] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:29 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:33:30.090703 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:30.101996 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee061f5-0aae-4424-8e3b-ec5471909457 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:30.107229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:30.107909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.108207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.108457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:30.109112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:30.110737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:30.122277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1217/2418] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:30.126525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:30.137781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbfec3-6703-4973-ad60-a69df052fbe9 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:30.143063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:30.143784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.144369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.144545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] PATH_INFO: `/v3/projects/39be5186c4ac4118a212e0d40cdfba0e/users/30c7252828ea4e89bc5504c53fe5c726/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:30.160142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39be5186c4ac4118a212e0d40cdfba0e, user_id=30c7252828ea4e89bc5504c53fe5c726, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:30.163029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:30.257874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1202/2419] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:30 2022] PUT /identity/v3/projects/39be5186c4ac4118a212e0d40cdfba0e/users/30c7252828ea4e89bc5504c53fe5c726/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:30.262899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:30.274671 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee0af3-217c-4372-be01-6bed0d379e47 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:30.339707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ab46b-e15c-4f9a-9eb9-033922a52252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:30.340478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ab46b-e15c-4f9a-9eb9-033922a52252 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.340792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ab46b-e15c-4f9a-9eb9-033922a52252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.341154 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ab46b-e15c-4f9a-9eb9-033922a52252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:30.341888 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ab46b-e15c-4f9a-9eb9-033922a52252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:30.343593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ab46b-e15c-4f9a-9eb9-033922a52252 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:30.355408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1218/2420] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:33:30.365578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:30.377055 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d42b8-f33c-4a6b-a7f6-894568d7db27 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:30.381946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:30.382636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.382910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.383187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] PATH_INFO: `/v3/projects/39be5186c4ac4118a212e0d40cdfba0e/users/30c7252828ea4e89bc5504c53fe5c726/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:30.385185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39be5186c4ac4118a212e0d40cdfba0e, user_id=30c7252828ea4e89bc5504c53fe5c726, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:30.387930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:30.399685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1203/2421] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:30 2022] PUT /identity/v3/projects/39be5186c4ac4118a212e0d40cdfba0e/users/30c7252828ea4e89bc5504c53fe5c726/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:30.411801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.412606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.412735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:30.447165 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] MFA Rules not processed for user `30c7252828ea4e89bc5504c53fe5c726`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:30.448477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:30.533839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1219/2422] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:30.543064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:30.545113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:30.641872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95643e0-bd88-4ea2-a1d9-b4cd007a287b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:30.645758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:30.646673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.647064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.647504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:30.658661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:30.660347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:30.671169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1204/2423] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:30.955911 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:30.966646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53c5b4-db97-46be-a0f4-fbae00ff152d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:30.970654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9242725-59d3-46ab-ae52-49bba317e7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:30.971252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9242725-59d3-46ab-ae52-49bba317e7ab service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:30.971539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9242725-59d3-46ab-ae52-49bba317e7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:30.971801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9242725-59d3-46ab-ae52-49bba317e7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:31.018984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:31.032976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9242725-59d3-46ab-ae52-49bba317e7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:31.034503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9242725-59d3-46ab-ae52-49bba317e7ab service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:31.045456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1220/2424] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:33:33.405858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:33.407403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:33.491748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3f141-580e-4fb4-9cc4-0591dae5d951 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:33.498602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:33.501066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:33.501565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:33.502039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:33.505436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:33.576733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:33.618572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:33.620135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471c047-9842-491b-a2d7-e4c71f2ae01e service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:33.632047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1205/2425] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:37.209221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39e2d-a546-4b16-8714-c6b459c399b3 None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:37.214379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39e2d-a546-4b16-8714-c6b459c399b3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:37.214379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a39e2d-a546-4b16-8714-c6b459c399b3 None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:37.214379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1221/2426] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:33:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:33:37.227222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91fad3-0c35-4983-acbf-9881699079b1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:37.227954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91fad3-0c35-4983-acbf-9881699079b1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:37.228244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91fad3-0c35-4983-acbf-9881699079b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:37.309548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-fc91fad3-0c35-4983-acbf-9881699079b1 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:37.311002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fc91fad3-0c35-4983-acbf-9881699079b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:37.429505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1206/2427] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:37.443134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39e2d-a546-4b16-8714-c6b459c399b3 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:37.446988 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-70a39e2d-a546-4b16-8714-c6b459c399b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:37.539277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a39e2d-a546-4b16-8714-c6b459c399b3 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:37.546158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603db87-0766-4f0c-b4c5-4317e3f2676d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:37.547824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603db87-0766-4f0c-b4c5-4317e3f2676d service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:37.548587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603db87-0766-4f0c-b4c5-4317e3f2676d service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:37.549213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603db87-0766-4f0c-b4c5-4317e3f2676d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:37.569184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603db87-0766-4f0c-b4c5-4317e3f2676d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:37.572021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603db87-0766-4f0c-b4c5-4317e3f2676d service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:37.595801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1222/2428] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:38.097753 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ae02b-a6b7-4444-bce4-e2c71309c245 None None] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.098665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ae02b-a6b7-4444-bce4-e2c71309c245 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.099257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ae02b-a6b7-4444-bce4-e2c71309c245 None None] PATH_INFO: `/` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.102301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1207/2429] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:33:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:33:38.109569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af008463-c043-4b40-97e1-2abd44017be4 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.110207 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af008463-c043-4b40-97e1-2abd44017be4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.110459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af008463-c043-4b40-97e1-2abd44017be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.134429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-af008463-c043-4b40-97e1-2abd44017be4 None None] MFA Rules not processed for user `73aebcec5ecc40ea9897fc7450c1cd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:38.134429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-af008463-c043-4b40-97e1-2abd44017be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:38.141088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1223/2430] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:33:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:38.160454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ae02b-a6b7-4444-bce4-e2c71309c245 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:38.162244 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-080ae02b-a6b7-4444-bce4-e2c71309c245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:38.176909 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ae02b-a6b7-4444-bce4-e2c71309c245 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:38.181109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db9be8-70a5-4932-a166-0c79b8029660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:38.181963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db9be8-70a5-4932-a166-0c79b8029660 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.182354 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db9be8-70a5-4932-a166-0c79b8029660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.182822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db9be8-70a5-4932-a166-0c79b8029660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.193920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db9be8-70a5-4932-a166-0c79b8029660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:38.197378 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db9be8-70a5-4932-a166-0c79b8029660 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:38.212516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1208/2431] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:38.413499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69436f-f769-4600-a08a-4a78fcca2ace None None] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.414817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69436f-f769-4600-a08a-4a78fcca2ace None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.417158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69436f-f769-4600-a08a-4a78fcca2ace None None] PATH_INFO: `/` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.421214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1224/2432] 10.222.0.65 () {54 vars in 921 bytes} [Mon Aug 29 21:33:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 21:33:38.427949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.429571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.429963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.458452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] MFA Rules not processed for user `3e5014eab4c7479f9ee493e31c91b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:38.459692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:38.470607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1209/2433] 10.222.0.65 () {58 vars in 1018 bytes} [Mon Aug 29 21:33:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:38.484454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69436f-f769-4600-a08a-4a78fcca2ace None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:38.503390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69436f-f769-4600-a08a-4a78fcca2ace None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:38.508380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:38.509025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.509305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.509579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.529007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:38.531164 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:38.545516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1225/2434] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:38.933239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:38.934912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:38.949868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4361d9-b0ce-4553-b621-a3450c0c91a8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:38.964950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:38.964950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:38.964950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:38.964950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] PATH_INFO: `/v3/users/d3c5edb153624912b87d61dd7ad5e4f3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:38.965466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c5edb153624912b87d61dd7ad5e4f3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:38.965466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:39.070829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:39.085947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ad116-fc91-4fb6-bdb2-2d3d07232d6e service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:39.091755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:39.092426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:39.092707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:39.093006 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] PATH_INFO: `/v3/users/cb355da653e04ec6ad48e8e66baf2f3f` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:39.094429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb355da653e04ec6ad48e8e66baf2f3f)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:39.096357 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:39.373527 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb355da653e04ec6ad48e8e66baf2f3f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.373879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c5edb153624912b87d61dd7ad5e4f3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.374695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c5edb153624912b87d61dd7ad5e4f3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.389734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c5edb153624912b87d61dd7ad5e4f3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.390607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c5edb153624912b87d61dd7ad5e4f3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.397975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c5edb153624912b87d61dd7ad5e4f3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.536456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb355da653e04ec6ad48e8e66baf2f3f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.549067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb355da653e04ec6ad48e8e66baf2f3f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.565333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb355da653e04ec6ad48e8e66baf2f3f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.566359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-74977f25-943a-4fbf-a0e2-fc2a3c3065fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb355da653e04ec6ad48e8e66baf2f3f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:39.573534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1226/2435] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:39 2022] DELETE /identity/v3/users/cb355da653e04ec6ad48e8e66baf2f3f => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:39.578283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:39.578574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:39.578876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:39.615222 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:39.616567 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:39.656465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1210/2436] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:38 2022] DELETE /identity/v3/users/d3c5edb153624912b87d61dd7ad5e4f3 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:39.668355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:39.706966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1227/2437] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:39.716499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:39.718683 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:39.749438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:39.803190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194b3bf-0103-4c5a-b80f-b21a280ad61d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:39.808607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9354eb-1d6f-45d1-95c9-07748c578132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:39.809241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9354eb-1d6f-45d1-95c9-07748c578132 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:39.809525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9354eb-1d6f-45d1-95c9-07748c578132 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:39.809802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9354eb-1d6f-45d1-95c9-07748c578132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:39.911718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9354eb-1d6f-45d1-95c9-07748c578132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:39.913405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9354eb-1d6f-45d1-95c9-07748c578132 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:39.934752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1211/2438] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:33:40.145554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:40.147199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:40.161407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46286d-bfba-4c88-a37f-16e67479ad0d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:40.168469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:40.168469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:40.168469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:40.168469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] PATH_INFO: `/v3/projects/d8d35a885c494e11b42f556554f3698b` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:40.169339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d35a885c494e11b42f556554f3698b)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:40.171401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:40.370799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d35a885c494e11b42f556554f3698b', 'request_id': 'req-f0b70c88-0d89-4c3b-a79a-8f33901d424c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.379754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d35a885c494e11b42f556554f3698b', 'request_id': 'req-f0b70c88-0d89-4c3b-a79a-8f33901d424c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.472414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d35a885c494e11b42f556554f3698b', 'request_id': 'req-f0b70c88-0d89-4c3b-a79a-8f33901d424c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.492984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] The token cache is being invalidate because project d8d35a885c494e11b42f556554f3698b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:40.493907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.495814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.504352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1228/2439] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:40 2022] DELETE /identity/v3/projects/d8d35a885c494e11b42f556554f3698b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:40.509440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b1db6-a785-4300-9f03-d2cfd413783f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:40.509440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b1db6-a785-4300-9f03-d2cfd413783f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:40.509440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b1db6-a785-4300-9f03-d2cfd413783f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:40.532259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f06b1db6-a785-4300-9f03-d2cfd413783f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:40.533471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f06b1db6-a785-4300-9f03-d2cfd413783f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:40.571729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:40.574151 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:40.620265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1212/2440] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:40.629998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:40.630279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:40.630800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:40.655199 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:40.656482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:40.669337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1213/2441] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:40.674113 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:40.685212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4077-cda3-4943-9357-0ae5d12483b6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:40.690150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:40.690788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:40.691064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:40.691358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:40.692026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:40.692334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:40.692958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b70c88-0d89-4c3b-a79a-8f33901d424c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:40.695109 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:40.701921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:40.703106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:40.703518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:40.704308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] PATH_INFO: `/v3/projects/64da9604bdf84d9cace5cf0c3c7bd067` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:40.706991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64da9604bdf84d9cace5cf0c3c7bd067)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:40.711229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1214/2442] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:33:40.714531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:40.718366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:40.731210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4571c-8816-4838-b783-37376bb1459c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:40.737907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:40.744538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:40.744538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:40.744538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:40.744538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:40.744538 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:40.884807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1215/2443] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:40 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:40.886066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:40.950604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64da9604bdf84d9cace5cf0c3c7bd067', 'request_id': 'req-ef171906-dc70-4f88-968b-33d7094d53e7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.958282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64da9604bdf84d9cace5cf0c3c7bd067', 'request_id': 'req-ef171906-dc70-4f88-968b-33d7094d53e7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:40.979813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63d79a-5d8f-4323-9e72-0c2aa6532f46 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.039701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.040486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.040793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.041048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.041721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.043727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.063245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64da9604bdf84d9cace5cf0c3c7bd067', 'request_id': 'req-ef171906-dc70-4f88-968b-33d7094d53e7'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.065602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:41.081764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] The token cache is being invalidate because project 64da9604bdf84d9cace5cf0c3c7bd067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:41.081764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.081764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.084340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1229/2444] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:40 2022] DELETE /identity/v3/projects/64da9604bdf84d9cace5cf0c3c7bd067 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:41.087213 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:41.090267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:41.116198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef171906-dc70-4f88-968b-33d7094d53e7 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.125190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.126713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.127155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.127636 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] PATH_INFO: `/v3/users/30c7252828ea4e89bc5504c53fe5c726` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.129902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c7252828ea4e89bc5504c53fe5c726)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.131488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:41.133109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.177484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1216/2445] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:40 2022] POST /identity/v3/users => generated 281 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:41.185474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:41.189334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:41.204934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd272180-2886-49ef-a781-efbf081fb932 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.210207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.210943 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.211432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.211791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.212560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.214309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.231846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1217/2446] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:41.234045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:41.247239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2bf70-f782-4cdf-bc75-ea417066040d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.255377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.255377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.255377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.255377 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.256335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.256600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.276891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1218/2447] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:41.281505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:41.293872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5cd09-8c23-4206-96e6-ebf8a0d6d734 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.299662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7b062-2411-4ece-a66d-074ec87f47e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.300477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7b062-2411-4ece-a66d-074ec87f47e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.300900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7b062-2411-4ece-a66d-074ec87f47e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.301265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7b062-2411-4ece-a66d-074ec87f47e5 admin admin] PATH_INFO: `/v3/projects/66d2969ddf8c43258a450e2bd9065037/users/15ff9fa9166e4f63963c0c3f2534ca8e/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.319793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7b062-2411-4ece-a66d-074ec87f47e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d2969ddf8c43258a450e2bd9065037, user_id=15ff9fa9166e4f63963c0c3f2534ca8e, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.323781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7b062-2411-4ece-a66d-074ec87f47e5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.394057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c7252828ea4e89bc5504c53fe5c726'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.419413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1219/2448] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:41 2022] PUT /identity/v3/projects/66d2969ddf8c43258a450e2bd9065037/users/15ff9fa9166e4f63963c0c3f2534ca8e/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:41.431076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22a10-5d3c-4457-871b-3a579ef7b33a None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.431363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22a10-5d3c-4457-871b-3a579ef7b33a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.431635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22a10-5d3c-4457-871b-3a579ef7b33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.462350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a2d22a10-5d3c-4457-871b-3a579ef7b33a None None] MFA Rules not processed for user `15ff9fa9166e4f63963c0c3f2534ca8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:41.464120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a2d22a10-5d3c-4457-871b-3a579ef7b33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:41.484873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c7252828ea4e89bc5504c53fe5c726'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.492960 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c7252828ea4e89bc5504c53fe5c726'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.507044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c7252828ea4e89bc5504c53fe5c726'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.508389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c7252828ea4e89bc5504c53fe5c726'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:41.515811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1230/2449] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:41 2022] DELETE /identity/v3/users/30c7252828ea4e89bc5504c53fe5c726 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:41.546708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1220/2450] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:41.602769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:41.604698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:41.667016 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f153a-93f9-43f5-be73-86d93518795b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.670834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.671854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.671854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.672104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.731692 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.733193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.734532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.734992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.735088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.742448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1231/2451] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:33:41.755499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:41.756748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:41.815366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1221/2452] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:41.829779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:41.831763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:41.924326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b74e8-bc4f-49e3-82c8-fbad09e9c692 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:41.929644 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a0597-b4c0-4395-affb-123294514fa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:41.930752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a0597-b4c0-4395-affb-123294514fa9 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:41.931040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a0597-b4c0-4395-affb-123294514fa9 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:41.931320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a0597-b4c0-4395-affb-123294514fa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:41.940743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a0597-b4c0-4395-affb-123294514fa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:41.942385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a0597-b4c0-4395-affb-123294514fa9 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:41.952113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1232/2453] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:33:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:33:42.296755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:42.298423 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:42.334138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:42.487552 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bae8eb-182a-4cd1-a9ef-60ea6ca823c1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:42.498546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:42.498546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:42.498770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:42.498887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] PATH_INFO: `/v3/projects/39be5186c4ac4118a212e0d40cdfba0e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:42.500545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39be5186c4ac4118a212e0d40cdfba0e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:42.507959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:42.603241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:42.680443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39be5186c4ac4118a212e0d40cdfba0e', 'request_id': 'req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:42.701570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39be5186c4ac4118a212e0d40cdfba0e', 'request_id': 'req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:42.787749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39be5186c4ac4118a212e0d40cdfba0e', 'request_id': 'req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:42.805783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] The token cache is being invalidate because project 39be5186c4ac4118a212e0d40cdfba0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:42.808072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:42.809323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-364aeb4c-aaf1-4b73-beb8-dcd8f000d1a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:42.811847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1222/2454] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:42 2022] DELETE /identity/v3/projects/39be5186c4ac4118a212e0d40cdfba0e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:42.817373 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:42.817756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:42.818125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:42.883133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:42.885827 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:43.021285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1233/2455] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:43.032230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:43.032672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:43.033241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:43.067329 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:43.071818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:43.085818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1223/2456] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:43.090442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:43.106049 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b6b56-7402-4f79-a320-4c21110599ae None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:43.114353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:43.117503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:43.117503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:43.117503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:43.117503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:43.117503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:43.120853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:43.142312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1234/2457] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:43.146693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:43.159148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705189e7-0773-4f88-8784-0aab9d88b66d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:43.166032 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:43.167777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:43.168048 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:43.168284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:43.169190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:43.171743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:43.293327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1224/2458] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:43 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:43.402077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:43.427123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c47ee4-7c28-46e1-b9b2-fb3c2afd0b2f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:43.585313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:43.586754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:43.587961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:43.589629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:43.591206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:43.596432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:43.827930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1235/2459] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:43 2022] POST /identity/v3/users => generated 286 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:43.834054 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:43.857200 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b68703-e902-4ccb-b0da-f145a96b2774 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:43.865687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:43.867248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:43.868149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:43.868816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:43.870441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:43.873747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:43.910756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1225/2460] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:43.919245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:43.939515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583398a-5c6f-4e5a-a3a0-71da7d68511b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:43.949675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cf33b-8d1b-4bd6-b9d7-5d3b61040bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:43.950908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf33b-8d1b-4bd6-b9d7-5d3b61040bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:43.951461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf33b-8d1b-4bd6-b9d7-5d3b61040bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:43.952061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cf33b-8d1b-4bd6-b9d7-5d3b61040bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:43.953307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cf33b-8d1b-4bd6-b9d7-5d3b61040bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:43.960978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508cf33b-8d1b-4bd6-b9d7-5d3b61040bd4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:43.980554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1236/2461] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:43.986383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:44.000343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:44.015102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e9d79-4d6a-4f7b-9191-81ec9f81fd5c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:44.033954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:44.041065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:44.041065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:44.041065 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] PATH_INFO: `/v3/projects/ace16be8fc9b4690bacc9f6cd507450e/users/f504cfed5fe0484fa0772efdcd5b5ebe/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:44.101578 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace16be8fc9b4690bacc9f6cd507450e, user_id=f504cfed5fe0484fa0772efdcd5b5ebe, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:44.119940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:44.256868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1226/2462] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:43 2022] PUT /identity/v3/projects/ace16be8fc9b4690bacc9f6cd507450e/users/f504cfed5fe0484fa0772efdcd5b5ebe/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:44.275464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:44.275464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:44.275464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:44.462740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] MFA Rules not processed for user `f504cfed5fe0484fa0772efdcd5b5ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:44.464923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:44.473672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:44.627268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1237/2463] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:44.650051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:44.653733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:44.745148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74502742-a4f3-4227-8db5-66c75db2e7ed admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:44.764340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d3e20-04ee-4dbf-b18e-09ee51ed31b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:44.765091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d3e20-04ee-4dbf-b18e-09ee51ed31b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:44.765526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d3e20-04ee-4dbf-b18e-09ee51ed31b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:44.765871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d3e20-04ee-4dbf-b18e-09ee51ed31b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:44.777206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d3e20-04ee-4dbf-b18e-09ee51ed31b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:44.778872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d3e20-04ee-4dbf-b18e-09ee51ed31b2 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:44.799634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1227/2464] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:44.830282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:44.965096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abdd78-10d8-46bb-b993-3db89163a35d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:44.965096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:44.965096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:44.965096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:44.965718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] PATH_INFO: `/v3/users/f504cfed5fe0484fa0772efdcd5b5ebe` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:44.965718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f504cfed5fe0484fa0772efdcd5b5ebe)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:44.965718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:45.438943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f504cfed5fe0484fa0772efdcd5b5ebe'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:45.680044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f504cfed5fe0484fa0772efdcd5b5ebe'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:45.705819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f504cfed5fe0484fa0772efdcd5b5ebe'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:45.731343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f504cfed5fe0484fa0772efdcd5b5ebe'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:45.732324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f504cfed5fe0484fa0772efdcd5b5ebe'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:45.744010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1238/2465] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:44 2022] DELETE /identity/v3/users/f504cfed5fe0484fa0772efdcd5b5ebe => generated 0 bytes in 913 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:45.748296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:45.748863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:45.749488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:45.792641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:45.795637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:45.924291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1228/2466] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:45.949513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:45.949513 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:46.005398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:46.086407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f0808-f517-4670-9307-a355b37c8c08 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:46.094770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b6c04-7e5e-43df-8a4d-70159cfc2d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:46.094770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6c04-7e5e-43df-8a4d-70159cfc2d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:46.094770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6c04-7e5e-43df-8a4d-70159cfc2d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:46.094770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b6c04-7e5e-43df-8a4d-70159cfc2d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:46.105849 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b6c04-7e5e-43df-8a4d-70159cfc2d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:46.107808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b6c04-7e5e-43df-8a4d-70159cfc2d19 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:46.121601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1239/2467] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:46.842073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:46.853290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:46.867089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68b76d-c169-421c-bc3b-86fac6e54d31 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:46.878075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:46.878736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:46.879019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:46.879298 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] PATH_INFO: `/v3/projects/ace16be8fc9b4690bacc9f6cd507450e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:46.880515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace16be8fc9b4690bacc9f6cd507450e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:46.882502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:47.216717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace16be8fc9b4690bacc9f6cd507450e', 'request_id': 'req-16f4b084-4dc4-429c-b8a0-3ef43c47c890'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:47.227680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace16be8fc9b4690bacc9f6cd507450e', 'request_id': 'req-16f4b084-4dc4-429c-b8a0-3ef43c47c890'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:47.323017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace16be8fc9b4690bacc9f6cd507450e', 'request_id': 'req-16f4b084-4dc4-429c-b8a0-3ef43c47c890'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:47.344242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] The token cache is being invalidate because project ace16be8fc9b4690bacc9f6cd507450e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:47.344782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:47.346224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-16f4b084-4dc4-429c-b8a0-3ef43c47c890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:47.355769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1229/2468] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:46 2022] DELETE /identity/v3/projects/ace16be8fc9b4690bacc9f6cd507450e => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:47.357741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:47.358542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:47.358542 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:47.387755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:47.389902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:47.506558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1240/2469] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:47.525918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:47.527008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:47.528127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:47.541620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:47.569193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:47.571941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:47.600020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1230/2470] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:47.608070 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:47.642820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3490-129f-4b4d-9059-c501a00edcc7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:47.668234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:47.672793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:47.673370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:47.673878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:47.675141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:47.675687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:47.688343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:47.721481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1241/2471] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:47.734744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:47.763306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a0845-1e50-45d0-8393-f952c4417e32 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:47.789659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:47.790692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:47.791877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:47.792532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:47.793823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:47.797797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:47.983956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1231/2472] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:47 2022] POST /identity/v3/projects => generated 359 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:47.990426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:48.014117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a26812-bab7-40df-993f-b095e96a13d5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:48.132230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:48.132230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:48.132230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:48.132230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:48.132969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:48.134573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:48.238187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1242/2473] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:47 2022] POST /identity/v3/users => generated 280 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:48.245003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:48.274404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3155-990f-4bfe-9bbb-7dadb6000c16 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:48.281643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:48.282445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:48.282805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:48.283207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:48.283994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:48.286127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:48.300837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1232/2474] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:48.310864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:48.329299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0f0c-3fd7-46d5-be21-6f95d6d462d1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:48.337819 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea16767-32c7-47e9-9bee-d43f3956fe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:48.339015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea16767-32c7-47e9-9bee-d43f3956fe5e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:48.339547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea16767-32c7-47e9-9bee-d43f3956fe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:48.340058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea16767-32c7-47e9-9bee-d43f3956fe5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:48.341159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea16767-32c7-47e9-9bee-d43f3956fe5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:48.344711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea16767-32c7-47e9-9bee-d43f3956fe5e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:48.370404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1243/2475] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:48.371441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:48.394875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71eb80-a3fe-47ef-827a-f60c3b59c2f0 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:48.401191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:48.401892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:48.402206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:48.402483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] PATH_INFO: `/v3/projects/ad5d6fb03b924a0cb040146a816265af/users/35f54d01fb5041ad960c2b258e3038d1/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:48.421996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad5d6fb03b924a0cb040146a816265af, user_id=35f54d01fb5041ad960c2b258e3038d1, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:48.424880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:48.506989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1233/2476] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:48 2022] PUT /identity/v3/projects/ad5d6fb03b924a0cb040146a816265af/users/35f54d01fb5041ad960c2b258e3038d1/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:48.520509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:48.521343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:48.522172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:48.564629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] MFA Rules not processed for user `35f54d01fb5041ad960c2b258e3038d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:48.565955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:48.595956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:48.669931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1244/2477] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:48.705728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:48.708380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:48.827381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff223-572d-422a-afe0-28c4847db72b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:48.845596 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfa7a9-8368-4687-826d-70a6f59e27bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:48.846440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfa7a9-8368-4687-826d-70a6f59e27bc service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:48.850715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfa7a9-8368-4687-826d-70a6f59e27bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:48.850715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfa7a9-8368-4687-826d-70a6f59e27bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:48.858455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfa7a9-8368-4687-826d-70a6f59e27bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:48.860398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfa7a9-8368-4687-826d-70a6f59e27bc service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:48.893652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1234/2478] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:48.917330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:49.022627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98144416-55e3-40c9-8fb9-8a1ca52fd975 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:49.029539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:49.029807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:49.030193 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:49.030751 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] PATH_INFO: `/v3/users/35f54d01fb5041ad960c2b258e3038d1` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:49.032517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f54d01fb5041ad960c2b258e3038d1)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:49.034814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:49.409099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f54d01fb5041ad960c2b258e3038d1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:49.788756 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f54d01fb5041ad960c2b258e3038d1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:49.816045 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f54d01fb5041ad960c2b258e3038d1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:49.837555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f54d01fb5041ad960c2b258e3038d1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:49.838963 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f54d01fb5041ad960c2b258e3038d1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:49.847978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1245/2479] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:48 2022] DELETE /identity/v3/users/35f54d01fb5041ad960c2b258e3038d1 => generated 0 bytes in 932 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:49.855096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:49.856016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:49.856582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:49.893446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:49.895751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:50.045889 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1235/2480] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:50.060161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:50.066074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:50.184411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:50.224908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23ef9c-28d4-4c5e-9cbc-4a7aef178a58 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:50.237112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:50.238304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:50.238881 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:50.239447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:50.271526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:50.273184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:50.286238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1246/2481] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:51.107620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:51.110052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:51.139709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee555de-bbdb-4b0d-9efd-3cf6fa4f6dfd None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:51.151854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:51.152575 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:51.152977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:51.153589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] PATH_INFO: `/v3/projects/ad5d6fb03b924a0cb040146a816265af` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:51.155034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad5d6fb03b924a0cb040146a816265af)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:51.157371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:51.332020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:51.341788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad5d6fb03b924a0cb040146a816265af', 'request_id': 'req-bd3664df-a7b0-438e-9078-f5973a495529'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:51.350704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad5d6fb03b924a0cb040146a816265af', 'request_id': 'req-bd3664df-a7b0-438e-9078-f5973a495529'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:51.452246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad5d6fb03b924a0cb040146a816265af', 'request_id': 'req-bd3664df-a7b0-438e-9078-f5973a495529'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:51.466882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] The token cache is being invalidate because project ad5d6fb03b924a0cb040146a816265af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:51.467330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:51.468075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd3664df-a7b0-438e-9078-f5973a495529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:51.469940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1236/2482] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:51 2022] DELETE /identity/v3/projects/ad5d6fb03b924a0cb040146a816265af => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:54.931050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:54.932711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:54.948234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac84a97-951e-45c9-9fce-c36a8dc374fd service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:55.017418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:55.018074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:55.018350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:55.018622 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] PATH_INFO: `/v3/users/15ff9fa9166e4f63963c0c3f2534ca8e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:55.019800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ff9fa9166e4f63963c0c3f2534ca8e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:55.022128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:56.187719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ff9fa9166e4f63963c0c3f2534ca8e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:56.308435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ff9fa9166e4f63963c0c3f2534ca8e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:56.317902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ff9fa9166e4f63963c0c3f2534ca8e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:56.332957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ff9fa9166e4f63963c0c3f2534ca8e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:56.333552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ff9fa9166e4f63963c0c3f2534ca8e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:56.341760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1247/2483] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:33:54 2022] DELETE /identity/v3/users/15ff9fa9166e4f63963c0c3f2534ca8e => generated 0 bytes in 1412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:33:56.347252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:56.348210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:56.348210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:56.379726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:56.381098 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:56.461458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1237/2484] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:56.473947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:56.476050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:56.567477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366de918-4f1e-4fc6-b808-aa89b4c12824 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:56.571845 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e09ad-f9d0-4683-9e51-a0fb532610dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:56.573933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e09ad-f9d0-4683-9e51-a0fb532610dd service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:56.574243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e09ad-f9d0-4683-9e51-a0fb532610dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:56.574562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e09ad-f9d0-4683-9e51-a0fb532610dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:56.589721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e09ad-f9d0-4683-9e51-a0fb532610dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:56.591657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e09ad-f9d0-4683-9e51-a0fb532610dd service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:56.609807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1248/2485] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:33:57.568867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:57.571018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:57.593372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ac75-4169-4f04-9933-8437cde4b5ec None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:57.600142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:57.601031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.601375 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.601789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] PATH_INFO: `/v3/projects/66d2969ddf8c43258a450e2bd9065037` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.604250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d2969ddf8c43258a450e2bd9065037)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:57.606324 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:57.808266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d2969ddf8c43258a450e2bd9065037', 'request_id': 'req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:57.816463 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d2969ddf8c43258a450e2bd9065037', 'request_id': 'req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:57.952936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d2969ddf8c43258a450e2bd9065037', 'request_id': 'req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:57.967959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] The token cache is being invalidate because project 66d2969ddf8c43258a450e2bd9065037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:33:57.968312 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:57.969206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-4ad0aa1b-4afc-44b9-a0c1-92fe2a98a132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:33:57.970845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1238/2486] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:33:57 2022] DELETE /identity/v3/projects/66d2969ddf8c43258a450e2bd9065037 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:57.976580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aea730-21bc-49fa-a390-24da724006bf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:57.977517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aea730-21bc-49fa-a390-24da724006bf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:57.977795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aea730-21bc-49fa-a390-24da724006bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:57.993562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.001519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-94aea730-21bc-49fa-a390-24da724006bf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:58.002774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-94aea730-21bc-49fa-a390-24da724006bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:58.073876 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1249/2487] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:58.090667 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.090933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.092770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.117216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:58.119278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:58.132237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1239/2488] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:58.138883 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aea730-21bc-49fa-a390-24da724006bf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.153820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aea730-21bc-49fa-a390-24da724006bf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.159470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.160146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.160431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.160707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.161416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:33:58.161726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.163570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.179468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1250/2489] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:33:58.184919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.195651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a126070-fb89-43f1-b658-3b42fd638724 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.200760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.201396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.201685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.201955 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.202663 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.204758 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.366821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1240/2490] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/projects => generated 401 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.371330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.382606 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c479fce-a90e-427e-a70e-c599a489250f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.440681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.441287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.441568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.441834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.442569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.444524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.580740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1251/2491] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/users => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:33:58.582108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.596978 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348e42a-1f8b-4280-b079-5504b3943f6d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.601961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.602595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.603016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.603267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.603968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.605624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.620271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1241/2492] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.621694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.633880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19a347-506e-4035-9f6e-d0371e983a51 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.642034 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbd369-ed71-48ac-8827-2af01e5b4898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.642678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbd369-ed71-48ac-8827-2af01e5b4898 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.642964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbd369-ed71-48ac-8827-2af01e5b4898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.643464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbd369-ed71-48ac-8827-2af01e5b4898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.644111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbd369-ed71-48ac-8827-2af01e5b4898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.645836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbd369-ed71-48ac-8827-2af01e5b4898 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.663120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1252/2493] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:33:58.664404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.675246 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb2927-c7e4-4c1d-b924-1fc4d1234f9a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.681233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c202c-1591-4eff-b33f-ea96d0223da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:58.682012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c202c-1591-4eff-b33f-ea96d0223da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.682179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c202c-1591-4eff-b33f-ea96d0223da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.682609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c202c-1591-4eff-b33f-ea96d0223da3 admin admin] PATH_INFO: `/v3/projects/f4880473c6e14f3899edecdf603cd25b/users/f3800ffa83954f8581da19f4a0949a23/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.692299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.706106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c202c-1591-4eff-b33f-ea96d0223da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4880473c6e14f3899edecdf603cd25b, user_id=f3800ffa83954f8581da19f4a0949a23, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:58.711460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c202c-1591-4eff-b33f-ea96d0223da3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:58.746120 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.748353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.748353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.768200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:58.769446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:58.831971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1253/2494] 10.222.0.65 () {58 vars in 967 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:33:58.846036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1242/2495] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:33:58 2022] PUT /identity/v3/projects/f4880473c6e14f3899edecdf603cd25b/users/f3800ffa83954f8581da19f4a0949a23/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:33:58.846509 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.847395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:58.850150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:58.850548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:58.850925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:58.875657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:33:58.880237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:33:58.881927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:58.941634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38586b84-38ef-4a81-a775-4fe2b3c21ae0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:58.949911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1243/2496] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:33:58.959937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:33:58.961383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:33:59.013479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.014166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.014422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.014698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.032117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef9607-e20f-4126-a503-a370352195b5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:33:59.035971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f917a2-8347-4157-8f1a-a49d4f764f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:33:59.036597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f917a2-8347-4157-8f1a-a49d4f764f27 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:33:59.036884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f917a2-8347-4157-8f1a-a49d4f764f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:33:59.037157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f917a2-8347-4157-8f1a-a49d4f764f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:33:59.047206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f917a2-8347-4157-8f1a-a49d4f764f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.048784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f917a2-8347-4157-8f1a-a49d4f764f27 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.060169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1244/2497] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:33:59.081820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:33:59.084224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:33:59.094690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1254/2498] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:33:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:34:01.096248 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.097095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.097662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.177520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] MFA Rules not processed for user `f3800ffa83954f8581da19f4a0949a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:01.179240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:01.289499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1245/2499] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:01.294631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.308555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07839-c79d-4763-8977-bfe2583e1135 service swift] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.316111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.316754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.317037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.317337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.318032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.320452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.434282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1255/2500] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:01 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:01.438518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.450832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a00726-0a48-4bcc-9a92-227015036dbc None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.471195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:01.528870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.529540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.529821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.530294 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.530950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.532997 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.658241 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1246/2501] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:01 2022] POST /identity/v3/users => generated 309 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:01.659952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.671474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3f418-6871-4953-8ac4-e4209219541f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.675675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.676325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.676926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.677374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.687147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.687147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.693761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1256/2502] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:01.698659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.719284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ff345-1b33-4a62-b95a-4fcf6eb7c8bc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.724015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.724689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.726950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.726950 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] PATH_INFO: `/v3/projects/36f26094c632460b81db80ca11a48611/users/074165016d8d48f8a9fbd8d9430329bf/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.742265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f26094c632460b81db80ca11a48611, user_id=074165016d8d48f8a9fbd8d9430329bf, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.745276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.863927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1247/2503] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:01 2022] PUT /identity/v3/projects/36f26094c632460b81db80ca11a48611/users/074165016d8d48f8a9fbd8d9430329bf/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:01.869078 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:01.888569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6dafe-381f-41ab-9c63-94a8a6e5c279 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:01.924313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:01.968691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86095033-9806-4076-b1cd-bd37aad6b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:01.969557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095033-9806-4076-b1cd-bd37aad6b8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:01.969836 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095033-9806-4076-b1cd-bd37aad6b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:01.970111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86095033-9806-4076-b1cd-bd37aad6b8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:01.970753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095033-9806-4076-b1cd-bd37aad6b8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:01.972403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86095033-9806-4076-b1cd-bd37aad6b8a6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:01.984441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1257/2504] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:01.988929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:02.000229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aa8be-9322-47f8-af9a-cd7d71d98703 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:02.005104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:02.005773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:02.006056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:02.006338 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] PATH_INFO: `/v3/projects/36f26094c632460b81db80ca11a48611/users/074165016d8d48f8a9fbd8d9430329bf/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:02.008310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f26094c632460b81db80ca11a48611, user_id=074165016d8d48f8a9fbd8d9430329bf, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:02.011717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:02.022977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1248/2505] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:01 2022] PUT /identity/v3/projects/36f26094c632460b81db80ca11a48611/users/074165016d8d48f8a9fbd8d9430329bf/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:03.815340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:03.816026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:03.816026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:03.855655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] MFA Rules not processed for user `074165016d8d48f8a9fbd8d9430329bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:03.855655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:03.936811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1258/2506] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:05.357874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:05.359623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:05.436341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdd242-07a5-48de-b699-754330a98849 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:05.442093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:05.442734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:05.443017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:05.443299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:05.445106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:05.457602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:05.459286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:05.470695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1249/2507] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:05.485936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:05.563318 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b74eb6-1f43-4efd-a55e-078ef46281ea None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:05.568609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:05.569634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:05.570578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:05.571096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] PATH_INFO: `/v3/users/f3800ffa83954f8581da19f4a0949a23` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:05.573790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3800ffa83954f8581da19f4a0949a23)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:05.580550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:05.864634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3800ffa83954f8581da19f4a0949a23'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.869248 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:05.987020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3800ffa83954f8581da19f4a0949a23'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:05.995977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3800ffa83954f8581da19f4a0949a23'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.008173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3800ffa83954f8581da19f4a0949a23'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.009562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3800ffa83954f8581da19f4a0949a23'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.015584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1259/2508] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:05 2022] DELETE /identity/v3/users/f3800ffa83954f8581da19f4a0949a23 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:06.326508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:06.339587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e466e6-e0e1-4847-93a3-009d6a2655a8 service placement] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:06.417231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:06.417231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:06.417231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:06.421370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] PATH_INFO: `/v3/projects/f4880473c6e14f3899edecdf603cd25b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:06.426584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4880473c6e14f3899edecdf603cd25b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:06.426584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:06.428315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:06.580353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4880473c6e14f3899edecdf603cd25b', 'request_id': 'req-2a65efa0-df53-4234-af01-9e1b8452f903'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.589279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4880473c6e14f3899edecdf603cd25b', 'request_id': 'req-2a65efa0-df53-4234-af01-9e1b8452f903'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.799009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4880473c6e14f3899edecdf603cd25b', 'request_id': 'req-2a65efa0-df53-4234-af01-9e1b8452f903'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.812456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] The token cache is being invalidate because project f4880473c6e14f3899edecdf603cd25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:06.812902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.813673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:06.815337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1250/2509] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:06 2022] DELETE /identity/v3/projects/f4880473c6e14f3899edecdf603cd25b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:06.819804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:06.821994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:06.849043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed2e2d-46f6-48a3-8d7b-c1929d8a9801 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:06.932558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:06.933225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:06.933499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:06.933774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] PATH_INFO: `/v3/users/074165016d8d48f8a9fbd8d9430329bf` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:06.935440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] RBAC: Authorizing `identity:delete_user(user_id=074165016d8d48f8a9fbd8d9430329bf)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:06.939042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:07.217499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074165016d8d48f8a9fbd8d9430329bf'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.330220 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074165016d8d48f8a9fbd8d9430329bf'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.338553 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074165016d8d48f8a9fbd8d9430329bf'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.355189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074165016d8d48f8a9fbd8d9430329bf'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.356085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074165016d8d48f8a9fbd8d9430329bf'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.362800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1260/2510] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:06 2022] DELETE /identity/v3/users/074165016d8d48f8a9fbd8d9430329bf => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:07.676070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:07.678391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:07.696208 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65efa0-df53-4234-af01-9e1b8452f903 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:07.762518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:07.763167 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:07.763447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:07.763721 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] PATH_INFO: `/v3/projects/36f26094c632460b81db80ca11a48611` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:07.764918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f26094c632460b81db80ca11a48611)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:07.766840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:07.915825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f26094c632460b81db80ca11a48611', 'request_id': 'req-f9273568-6295-4e34-869b-bf476295b899'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:07.927923 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f26094c632460b81db80ca11a48611', 'request_id': 'req-f9273568-6295-4e34-869b-bf476295b899'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:08.023757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f26094c632460b81db80ca11a48611', 'request_id': 'req-f9273568-6295-4e34-869b-bf476295b899'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:08.028208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.029122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:08.036568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] The token cache is being invalidate because project 36f26094c632460b81db80ca11a48611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:08.036948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:08.037701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-f9273568-6295-4e34-869b-bf476295b899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:08.039424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1251/2511] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:07 2022] DELETE /identity/v3/projects/36f26094c632460b81db80ca11a48611 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:08.044994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f1964-c566-45ae-8748-954d419cad26 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.045374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f1964-c566-45ae-8748-954d419cad26 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.045714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f1964-c566-45ae-8748-954d419cad26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.073715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6a6f1964-c566-45ae-8748-954d419cad26 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:08.075058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6a6f1964-c566-45ae-8748-954d419cad26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:08.089801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:08.094628 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4aab8-738f-4049-9ec1-3359bb18563e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.095242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:08.151705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1252/2512] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:08.152805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.153445 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.153729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.154002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.155826 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:08.159716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.160043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.160540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.169940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.171638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.184115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1261/2513] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:08.190843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:08.193380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:08.207373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1253/2514] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:08.211480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.223650 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc0ecd-8a9e-4076-8069-f1f7a10f593e service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.230780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.231349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.231750 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.232196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.233003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:08.233446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.235414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.249598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1262/2515] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:34:08.255184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.267574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07321576-5c06-4956-9cd1-0acbd57445dc None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.275945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.276680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.276972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.277264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.277941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.279942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.430680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1254/2516] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/projects => generated 391 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:08.437480 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.450115 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c44e81-fa37-4e5f-8510-fe3a1e8c3241 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.531951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.532909 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.537955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.537955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.537955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.538953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.694289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1263/2517] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/users => generated 296 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:08.697862 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.714346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478d823-e32a-4848-8d4f-0076c0a33dba admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.720300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.721493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.721994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.722513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.723635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.726308 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.744861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1255/2518] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:08.749658 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.764266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c1920-c9bf-45fb-8f5a-fd4dc6231782 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.770999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55deea8-aa8f-4473-adc7-373cce83cc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.770999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55deea8-aa8f-4473-adc7-373cce83cc2b admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.770999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55deea8-aa8f-4473-adc7-373cce83cc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.770999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55deea8-aa8f-4473-adc7-373cce83cc2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.776333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55deea8-aa8f-4473-adc7-373cce83cc2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.776333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55deea8-aa8f-4473-adc7-373cce83cc2b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.786951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1264/2519] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:08.791750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:08.803477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15c22-89bc-4fdc-a309-168525307c6a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:08.808568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:08.809249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.809524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.809798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] PATH_INFO: `/v3/projects/faacc8a86fea4712adff7e77c31675b2/users/db7a3beb46ae4678956493f9d854ac3a/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.825905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faacc8a86fea4712adff7e77c31675b2, user_id=db7a3beb46ae4678956493f9d854ac3a, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:08.828780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:08.951761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1256/2520] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:08 2022] PUT /identity/v3/projects/faacc8a86fea4712adff7e77c31675b2/users/db7a3beb46ae4678956493f9d854ac3a/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:08.964343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:08.964743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:08.965111 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:08.998374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] MFA Rules not processed for user `db7a3beb46ae4678956493f9d854ac3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:08.999900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:09.089206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1265/2521] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:09.094193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.174503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a7e77-1422-4099-b1c4-b7941e619927 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.180197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.181462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.181799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.182096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.182897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.184917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.319330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1257/2522] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/projects => generated 393 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:09.323549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.331374 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:09.337829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66868bd6-0bd5-43b9-a1b3-5e7bccf6d45e None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.419015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.419653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.419984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.420272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.420922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.422904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.613685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1266/2523] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/users => generated 303 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:09.615269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.627154 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61cc969-17da-4813-a630-38f597b01eb7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.632004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.632630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.632908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.633178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.634533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.636218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.649085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1258/2524] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:09.653482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.665180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6a7a8-a627-4b05-97c5-aa4c7f7857a3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.669946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.670686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.671003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.671276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] PATH_INFO: `/v3/projects/c3bf48c8780f4b64ae55627de2b0415b/users/55557f65347b413687190b69c5f5b9ce/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.691570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bf48c8780f4b64ae55627de2b0415b, user_id=55557f65347b413687190b69c5f5b9ce, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.695031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.833983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1267/2525] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:09 2022] PUT /identity/v3/projects/c3bf48c8780f4b64ae55627de2b0415b/users/55557f65347b413687190b69c5f5b9ce/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:09.835194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.847144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025692fe-667e-4df2-92db-9aba470bbe9e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.885042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:09.920844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e796a0-5508-41ce-9cca-e9fa95adf122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.921492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e796a0-5508-41ce-9cca-e9fa95adf122 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.921774 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e796a0-5508-41ce-9cca-e9fa95adf122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.922063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e796a0-5508-41ce-9cca-e9fa95adf122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.922913 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e796a0-5508-41ce-9cca-e9fa95adf122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.924916 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e796a0-5508-41ce-9cca-e9fa95adf122 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.939205 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1259/2526] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:09.944162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:09.957033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30141e3c-3732-40d5-a4a2-807ec02412f8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:09.962511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:09.963312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.963665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.964064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] PATH_INFO: `/v3/projects/c3bf48c8780f4b64ae55627de2b0415b/users/55557f65347b413687190b69c5f5b9ce/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:09.970015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bf48c8780f4b64ae55627de2b0415b, user_id=55557f65347b413687190b69c5f5b9ce, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:09.970015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:09.982806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1268/2527] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:09 2022] PUT /identity/v3/projects/c3bf48c8780f4b64ae55627de2b0415b/users/55557f65347b413687190b69c5f5b9ce/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:09.994189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:09.994562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:09.995086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.044251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] MFA Rules not processed for user `55557f65347b413687190b69c5f5b9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:10.045933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:10.175434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1260/2528] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:10.185360 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.188508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:10.279354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba15c5-6e00-4807-a2f5-5309966fd01f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.283538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.284348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.284526 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.284773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.299053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.300643 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.312816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1269/2529] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:10 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:10.396255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:10.397704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:10.413565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d3924-f61d-4bbb-8b78-08260a486085 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:10.419868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:10.420958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:10.421760 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:10.421975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:10.511588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:10.513490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:10.526045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1261/2530] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:12.643026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:12.644538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:12.712424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:12.717144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d37fc-517b-4167-a8f7-ebbc8f1ef786 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:12.722609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:12.723443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:12.723725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:12.724042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] PATH_INFO: `/v3/users/68c15654f5a14850a1763a020c17c849` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:12.725854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c15654f5a14850a1763a020c17c849)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:12.728378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:12.968348 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c15654f5a14850a1763a020c17c849'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.067155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c15654f5a14850a1763a020c17c849'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.074866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c15654f5a14850a1763a020c17c849'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.090279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c15654f5a14850a1763a020c17c849'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.091341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-adb24646-8696-47cf-b195-813f26ba01ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c15654f5a14850a1763a020c17c849'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.101379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1270/2531] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:12 2022] DELETE /identity/v3/users/68c15654f5a14850a1763a020c17c849 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:13.445946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:13.459018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a40f6-e862-42f9-bcd3-9fe92afe1ecc service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:13.511337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=100 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:13.521012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:13.521610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.521894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.522237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] PATH_INFO: `/v3/projects/8695633e1e2744b6b6554536f1109fc6` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.523295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8695633e1e2744b6b6554536f1109fc6)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.525771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.681119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8695633e1e2744b6b6554536f1109fc6', 'request_id': 'req-60ba9b58-4c85-4465-b9a6-a1d2de960a59'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.687692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8695633e1e2744b6b6554536f1109fc6', 'request_id': 'req-60ba9b58-4c85-4465-b9a6-a1d2de960a59'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.801664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8695633e1e2744b6b6554536f1109fc6', 'request_id': 'req-60ba9b58-4c85-4465-b9a6-a1d2de960a59'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.812942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] The token cache is being invalidate because project 8695633e1e2744b6b6554536f1109fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:13.813234 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.813891 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-60ba9b58-4c85-4465-b9a6-a1d2de960a59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:13.815409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1262/2532] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:13 2022] DELETE /identity/v3/projects/8695633e1e2744b6b6554536f1109fc6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:13.820752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.821790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.821790 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.844291 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:13.845594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:13.913227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1271/2533] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:13.921647 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.921937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.922218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.944218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:13.945457 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:13.955373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1263/2534] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:13.960185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:13.970998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b4f1e-bb9a-4eb1-b48a-a5ac4cd373b2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:13.976202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:13.976841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:13.977162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:13.977451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:13.978285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:13.978593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:13.980327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:13.994698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1272/2535] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:13.999456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.010649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924a688-7ff0-4f96-9574-96cbc747c55e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.016001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.016634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.017001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.017277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.017942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.019986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.155314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1264/2536] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:13 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:14.159668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.170762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3877c6-adb7-418a-b9e6-512c3500b460 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.230695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.231315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.231594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.231869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.232539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.234515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.245824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:14.392742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1273/2537] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:14 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:14.396912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.407175 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d0479-def1-4f2d-b362-a50397a2b6d5 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.412201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.412878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.413185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.413440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.414121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.416273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.426533 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1265/2538] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:14.430673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.441475 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7695b4-0270-4421-b23d-5573752ac0c9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.447688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98494c-7a24-43d4-bd3b-9e01e90c1d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.448325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98494c-7a24-43d4-bd3b-9e01e90c1d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.448604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98494c-7a24-43d4-bd3b-9e01e90c1d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.448880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98494c-7a24-43d4-bd3b-9e01e90c1d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.449517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98494c-7a24-43d4-bd3b-9e01e90c1d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.451099 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98494c-7a24-43d4-bd3b-9e01e90c1d86 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.461924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1274/2539] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:14.466362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.477311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9223-5308-4b25-8972-056fc73ecd82 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.482480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.483257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.483540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.483816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] PATH_INFO: `/v3/projects/cd619246aefb4524ac238a8a7afde7cf/users/106dfde76b7b4e15918ec31efb416c43/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.503414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd619246aefb4524ac238a8a7afde7cf, user_id=106dfde76b7b4e15918ec31efb416c43, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.506500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.623947 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1266/2540] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:14 2022] PUT /identity/v3/projects/cd619246aefb4524ac238a8a7afde7cf/users/106dfde76b7b4e15918ec31efb416c43/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:14.635924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.636514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.637015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.671281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] MFA Rules not processed for user `106dfde76b7b4e15918ec31efb416c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:14.672484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:14.740266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1275/2541] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:14.752370 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:14.754075 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:14.839792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5eba02-b7a6-4f32-a378-84b4fd5d1492 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:14.846530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10052b2-11af-48c5-8bd9-8ac3b3a91d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:14.847173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10052b2-11af-48c5-8bd9-8ac3b3a91d58 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:14.847451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10052b2-11af-48c5-8bd9-8ac3b3a91d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:14.847737 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10052b2-11af-48c5-8bd9-8ac3b3a91d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:14.858239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10052b2-11af-48c5-8bd9-8ac3b3a91d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:14.860229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10052b2-11af-48c5-8bd9-8ac3b3a91d58 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:14.870436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1267/2542] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:15.132806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:15.215051 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b363b-4825-42cc-9234-0911acbbc3b0 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:15.219959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:15.224093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:15.224093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:15.224093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] PATH_INFO: `/v3/users/106dfde76b7b4e15918ec31efb416c43` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:15.224093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=106dfde76b7b4e15918ec31efb416c43)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:15.224504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:15.469716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106dfde76b7b4e15918ec31efb416c43'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.575789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106dfde76b7b4e15918ec31efb416c43'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.583773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106dfde76b7b4e15918ec31efb416c43'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.592423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:15.598007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106dfde76b7b4e15918ec31efb416c43'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.598901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106dfde76b7b4e15918ec31efb416c43'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:15.605855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1276/2543] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:15 2022] DELETE /identity/v3/users/106dfde76b7b4e15918ec31efb416c43 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:15.611583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:15.612320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:15.612320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:15.621094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:15.633530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:15.641567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:15.708328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1268/2544] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:15.727637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:15.730856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:15.801689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1a18c-912b-4a78-a0c9-4c4afe50dbf5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:15.805422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:15.806055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:15.806998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:15.807277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:15.817761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:15.819236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:15.832319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1277/2545] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:16.417372 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.420931 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.435963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169567f-eb56-45a0-b8f5-f6ecae513906 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.443379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.444572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.445084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.445989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] PATH_INFO: `/v3/projects/cd619246aefb4524ac238a8a7afde7cf` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.448271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd619246aefb4524ac238a8a7afde7cf)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.456105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.576218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.577688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.591664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e546-b3d7-4c3f-b10d-61ca36f6b101 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.606446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd619246aefb4524ac238a8a7afde7cf', 'request_id': 'req-0b23130a-9c4b-46f1-977c-1c3ee9406e67'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.613995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd619246aefb4524ac238a8a7afde7cf', 'request_id': 'req-0b23130a-9c4b-46f1-977c-1c3ee9406e67'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.656284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.656938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.657217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.657487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] PATH_INFO: `/v3/users/db7a3beb46ae4678956493f9d854ac3a` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.658672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7a3beb46ae4678956493f9d854ac3a)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.660906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.710432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd619246aefb4524ac238a8a7afde7cf', 'request_id': 'req-0b23130a-9c4b-46f1-977c-1c3ee9406e67'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.725550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] The token cache is being invalidate because project cd619246aefb4524ac238a8a7afde7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:16.725843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.726812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0b23130a-9c4b-46f1-977c-1c3ee9406e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.728877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1269/2546] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:16 2022] DELETE /identity/v3/projects/cd619246aefb4524ac238a8a7afde7cf => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:16.734620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d6aa0-8ff8-45de-b679-e05fc2682bc5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.734971 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d6aa0-8ff8-45de-b679-e05fc2682bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.735306 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d6aa0-8ff8-45de-b679-e05fc2682bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.757400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4e3d6aa0-8ff8-45de-b679-e05fc2682bc5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:16.760020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4e3d6aa0-8ff8-45de-b679-e05fc2682bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.767117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1270/2547] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:16.776745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.777023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.777296 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.797510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:16.798704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:16.807612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1271/2548] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:16.811876 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.821436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c84b8f-b7df-444e-8ac6-bb86411abd80 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.826115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.826728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.827004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.827355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.828061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:16.828436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.831523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.844573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1272/2549] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:34:16.849953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:16.860980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4aad4-f7e0-4e37-a9e2-876f1ad4fa63 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:16.865751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:16.866367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:16.866646 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:16.866928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:16.867550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:16.870185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:16.940914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7a3beb46ae4678956493f9d854ac3a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:16.967544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1273/2550] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:16.971793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.039384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:17.043679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7a3beb46ae4678956493f9d854ac3a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.051252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7a3beb46ae4678956493f9d854ac3a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.060492 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f042ae-f951-471a-a9d1-7eafd43bf0c3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.066529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7a3beb46ae4678956493f9d854ac3a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.067365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-68a650fc-cf4c-4b72-bc7e-aaf425953c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7a3beb46ae4678956493f9d854ac3a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:17.069239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.069881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.070172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.070438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.071080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.074019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.078761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1278/2551] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:16 2022] DELETE /identity/v3/users/db7a3beb46ae4678956493f9d854ac3a => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:17.084887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.085886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.085886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.106631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:17.107865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.116307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:17.177738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1279/2552] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:17.189321 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.190725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.220750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1274/2553] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:16 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:17.220750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.222702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.235859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fae9a-8008-47f8-86e9-297c1abbdabe admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.240814 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.241431 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.241715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.242089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.243263 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.245755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.258368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1275/2554] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:17.271139 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.272455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769b84a-090a-448a-855f-6f2179ea543b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.278707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.279773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.280290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.280752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.281946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693767a-3c82-4ce2-82e8-1bda3d5fe604 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.288189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef6140-2d8f-4739-a52c-3afd93586b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.288826 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6140-2d8f-4739-a52c-3afd93586b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.289129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6140-2d8f-4739-a52c-3afd93586b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.289405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef6140-2d8f-4739-a52c-3afd93586b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.290057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef6140-2d8f-4739-a52c-3afd93586b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.293432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef6140-2d8f-4739-a52c-3afd93586b5b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.293684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.295326 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.306972 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1280/2555] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:17.316337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1276/2556] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:17.321204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.332969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0ad05-756b-4fd3-a61b-d15e0bbb7886 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.339234 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.340076 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.340421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.340772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] PATH_INFO: `/v3/projects/e3effb7cff314875b00b9f11b7a1e94a/users/feca3de18907429694445faecedd083a/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.362672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3effb7cff314875b00b9f11b7a1e94a, user_id=feca3de18907429694445faecedd083a, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.365561 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.491431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1281/2557] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:17 2022] PUT /identity/v3/projects/e3effb7cff314875b00b9f11b7a1e94a/users/feca3de18907429694445faecedd083a/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:17.503124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.503539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.504512 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.536942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] MFA Rules not processed for user `feca3de18907429694445faecedd083a`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:17.538193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.628071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1277/2558] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:17.633347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.728821 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb957f-7aa5-4bc6-ba9e-8ed2d9f688a0 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.737665 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.741597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.741597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.741597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.741597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.745428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.889335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.891643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:17.916339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c24cd-48ac-4ba1-9717-0e2fa5230d1d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:17.921260 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:17.921912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:17.922197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:17.922459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] PATH_INFO: `/v3/projects/faacc8a86fea4712adff7e77c31675b2` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:17.923829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faacc8a86fea4712adff7e77c31675b2)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:17.925773 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:17.938039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1282/2559] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:17 2022] POST /identity/v3/projects => generated 373 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:17.942862 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:17.972140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:18.030654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62c2b9-d2f9-44d2-ae73-effaa1b3bd60 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.035669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.036492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.036776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.037048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.037702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.039904 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.197921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faacc8a86fea4712adff7e77c31675b2', 'request_id': 'req-d9438996-2924-43a6-aab9-0c1751fc1c32'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.205534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faacc8a86fea4712adff7e77c31675b2', 'request_id': 'req-d9438996-2924-43a6-aab9-0c1751fc1c32'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.374566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1283/2560] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:17 2022] POST /identity/v3/users => generated 293 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:18.379319 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.401218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a50f2-8ba7-4411-ba65-709bbfd4cd66 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.449311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.449922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.450205 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.450473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.451110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.452757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.462598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1284/2561] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:18.466630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.476427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e589cf-9865-4430-a1d5-01f6ce2514c4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.481270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.482005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.482310 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.482612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] PATH_INFO: `/v3/projects/2d07ce1b0727438f84c3c1096eff345a/users/d9cc506f8cfb4693bc036f051d2cc7e5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.499983 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d07ce1b0727438f84c3c1096eff345a, user_id=d9cc506f8cfb4693bc036f051d2cc7e5, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.502689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.523572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faacc8a86fea4712adff7e77c31675b2', 'request_id': 'req-d9438996-2924-43a6-aab9-0c1751fc1c32'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.535000 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] The token cache is being invalidate because project faacc8a86fea4712adff7e77c31675b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:18.535295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.535993 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.537543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1278/2562] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:17 2022] DELETE /identity/v3/projects/faacc8a86fea4712adff7e77c31675b2 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:18.541795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.543302 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.557685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9438996-2924-43a6-aab9-0c1751fc1c32 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.562594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.563319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.563594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.563869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] PATH_INFO: `/v3/users/55557f65347b413687190b69c5f5b9ce` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.565087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=55557f65347b413687190b69c5f5b9ce)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.567195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.584187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:18.644456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1285/2563] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:18 2022] PUT /identity/v3/projects/2d07ce1b0727438f84c3c1096eff345a/users/d9cc506f8cfb4693bc036f051d2cc7e5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:18.648266 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.650059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.729619 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c971ec-dece-4b2f-b34d-8b6f6e66b4ac admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.735026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.735698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.736007 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.736284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.736927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.738528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.754882 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1286/2564] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:18.760181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.774392 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d12ae5-66c8-45fc-9815-3505dec93ef2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:18.782303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c4031-b18b-48eb-9307-f21c550f0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:18.783450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c4031-b18b-48eb-9307-f21c550f0977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.783999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c4031-b18b-48eb-9307-f21c550f0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.784495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c4031-b18b-48eb-9307-f21c550f0977 admin admin] PATH_INFO: `/v3/projects/2d07ce1b0727438f84c3c1096eff345a/users/d9cc506f8cfb4693bc036f051d2cc7e5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.787795 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c4031-b18b-48eb-9307-f21c550f0977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d07ce1b0727438f84c3c1096eff345a, user_id=d9cc506f8cfb4693bc036f051d2cc7e5, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:18.793177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c4031-b18b-48eb-9307-f21c550f0977 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:18.808155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1287/2565] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:18 2022] PUT /identity/v3/projects/2d07ce1b0727438f84c3c1096eff345a/users/d9cc506f8cfb4693bc036f051d2cc7e5/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:18.819872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55557f65347b413687190b69c5f5b9ce'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.820505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:18.820505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:18.820745 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:18.820954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55557f65347b413687190b69c5f5b9ce'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.834121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55557f65347b413687190b69c5f5b9ce'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.836995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55557f65347b413687190b69c5f5b9ce'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.842831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55557f65347b413687190b69c5f5b9ce'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:18.851914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] MFA Rules not processed for user `d9cc506f8cfb4693bc036f051d2cc7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:18.854170 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:18.940919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1288/2566] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:18.953761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:18.955169 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.008245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1279/2567] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:18 2022] DELETE /identity/v3/users/55557f65347b413687190b69c5f5b9ce => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:19.037784 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20555d3d-af8c-423a-8871-10cf930aadcb None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:19.101069 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:19.102094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.103402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.103907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.176990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:19.179660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:19.181798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:19.192763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1289/2568] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:19.517977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:19.529045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c436f-392c-4bd7-bbb5-80159fcc2c8a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:19.532672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:19.533273 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.533549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.533819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.535509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.571478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:19.573592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:19.615129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:19.616636 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6a2c-bb0d-4187-b174-cb6ed0112c07 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:19.626581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1280/2569] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:19.653236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f91d71-f7ad-47c3-aafd-5b24a1057a06 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:19.659704 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:19.660709 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.661250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.661832 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] PATH_INFO: `/v3/projects/c3bf48c8780f4b64ae55627de2b0415b` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.663739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3bf48c8780f4b64ae55627de2b0415b)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:19.667087 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:19.670282 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:19.818015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3bf48c8780f4b64ae55627de2b0415b', 'request_id': 'req-bd3f8a2c-d20a-4672-a038-2d22dd036288'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.827872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3bf48c8780f4b64ae55627de2b0415b', 'request_id': 'req-bd3f8a2c-d20a-4672-a038-2d22dd036288'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.924211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3bf48c8780f4b64ae55627de2b0415b', 'request_id': 'req-bd3f8a2c-d20a-4672-a038-2d22dd036288'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.939468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] The token cache is being invalidate because project c3bf48c8780f4b64ae55627de2b0415b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:19.939760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.940474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bd3f8a2c-d20a-4672-a038-2d22dd036288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:19.942063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1290/2570] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:19 2022] DELETE /identity/v3/projects/c3bf48c8780f4b64ae55627de2b0415b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:19.947658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:19.948269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:19.948269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:19.969562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:19.970865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:20.038018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1281/2571] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:20.047260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.047725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.049037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.085013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:20.086595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:20.094890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1291/2572] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:20.099572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.109786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a7f3-c594-454c-958e-b7df50d63738 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.114472 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.117680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.117680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.117680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.117680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:20.117680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.118317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.135624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1282/2573] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:20.140333 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.150721 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551002-0eeb-42f2-8477-7e4687c6e242 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.155733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.156600 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.156884 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.164979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.165707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.167778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.320289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1292/2574] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:20.326681 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.345355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a85c8-03a4-43d7-8a9e-282f89a8b603 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.360766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:20.410481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.411387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.411695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.412060 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.412700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.414662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.528693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1283/2575] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:20.532646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.543778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf7a7e-c871-4766-91df-e7f2f3d06e19 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.548748 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.549379 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.549666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.549945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.550837 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.552304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.566804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1293/2576] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:20.571043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.581190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a13622-e379-4a9a-b76c-2a701740f128 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.586319 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a24ee5-a108-4e32-b954-5f469c6a8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.586935 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24ee5-a108-4e32-b954-5f469c6a8b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.587216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24ee5-a108-4e32-b954-5f469c6a8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.587523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24ee5-a108-4e32-b954-5f469c6a8b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.588262 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a24ee5-a108-4e32-b954-5f469c6a8b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.589903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a24ee5-a108-4e32-b954-5f469c6a8b05 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.602652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1284/2577] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:20.607833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.619875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1e77a-391f-453b-8f31-61fae7a841ae admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.624711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.625380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.625661 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.625938 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] PATH_INFO: `/v3/projects/85a7a08436a94038af6a7606a87ad96b/users/72703673ad964c42967fe239bad421ee/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.644221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a7a08436a94038af6a7606a87ad96b, user_id=72703673ad964c42967fe239bad421ee, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.647124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:20.727415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1294/2578] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:20 2022] PUT /identity/v3/projects/85a7a08436a94038af6a7606a87ad96b/users/72703673ad964c42967fe239bad421ee/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:20.739081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.739346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.739625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.769451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] MFA Rules not processed for user `72703673ad964c42967fe239bad421ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:20.770669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:20.841502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1285/2579] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:20.846425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:20.869552 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:20.915451 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4991e2-cebd-4ae6-aebb-35404ea241e8 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:20.920467 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:20.921143 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:20.921421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:20.921698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:20.922351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:20.924329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.108194 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1295/2580] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:20 2022] POST /identity/v3/projects => generated 387 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:21.112505 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.127033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dd50b-64ca-484d-aaec-ac1fc2209217 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.192444 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.193091 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.193371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.193649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.194350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.196402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.327559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1286/2581] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:21 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:34:21.332267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.346110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91d62c-d385-4027-a26d-85c21b242208 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.353769 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.354666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.354945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.355216 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.355897 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.357605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.369616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1296/2582] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:21.376771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.389412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac374263-e139-4ea9-a84a-674223e6f50f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.394163 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.394835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.395110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.395383 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] PATH_INFO: `/v3/projects/de7bd67abc8b4de3bb223db063b7e63e/users/6b083a07f4614623976b479a722fd207/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.412285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:21.413546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7bd67abc8b4de3bb223db063b7e63e, user_id=6b083a07f4614623976b479a722fd207, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.417616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.564881 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1287/2583] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:21 2022] PUT /identity/v3/projects/de7bd67abc8b4de3bb223db063b7e63e/users/6b083a07f4614623976b479a722fd207/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:21.569338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.583179 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310006b-b4c1-4e5a-a7d7-0c45c79326d3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.654057 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dd2d9-c7a2-48c7-ac28-cc6825083b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.654742 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dd2d9-c7a2-48c7-ac28-cc6825083b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.655039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dd2d9-c7a2-48c7-ac28-cc6825083b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.655398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dd2d9-c7a2-48c7-ac28-cc6825083b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.656278 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dd2d9-c7a2-48c7-ac28-cc6825083b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.658140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dd2d9-c7a2-48c7-ac28-cc6825083b21 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.669530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1297/2584] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:21.673929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.684979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a0c033-0521-4e76-a785-10647fc666e4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.692201 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.692893 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.693178 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.693455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] PATH_INFO: `/v3/projects/de7bd67abc8b4de3bb223db063b7e63e/users/6b083a07f4614623976b479a722fd207/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.695437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] RBAC: Authorizing `identity:create_grant(project_id=de7bd67abc8b4de3bb223db063b7e63e, user_id=6b083a07f4614623976b479a722fd207, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.698211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.709481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1288/2585] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:21 2022] PUT /identity/v3/projects/de7bd67abc8b4de3bb223db063b7e63e/users/6b083a07f4614623976b479a722fd207/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:21.720641 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.720934 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.721221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.751231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] MFA Rules not processed for user `6b083a07f4614623976b479a722fd207`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:21.753117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:21.832603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1298/2586] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:21.844777 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:21.846537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:21.913460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c3a58d-cd9d-402a-9f89-a1f34449fabf admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:21.918582 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab3108-3041-4244-804c-1e14128074eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:21.919402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3108-3041-4244-804c-1e14128074eb service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:21.920621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3108-3041-4244-804c-1e14128074eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:21.920897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3108-3041-4244-804c-1e14128074eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:21.931376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab3108-3041-4244-804c-1e14128074eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:21.933052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab3108-3041-4244-804c-1e14128074eb service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:21.943458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1289/2587] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:23.722726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:23.746380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:23.803621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8d1bf-eea0-454b-9348-38fb6adfecac None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:23.808334 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:23.808994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:23.809284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:23.809563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] PATH_INFO: `/v3/users/72703673ad964c42967fe239bad421ee` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:23.810732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=72703673ad964c42967fe239bad421ee)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:23.812740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:24.186694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72703673ad964c42967fe239bad421ee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:24.295584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72703673ad964c42967fe239bad421ee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:24.304315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72703673ad964c42967fe239bad421ee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:24.316415 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72703673ad964c42967fe239bad421ee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:24.317185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72703673ad964c42967fe239bad421ee'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:24.325041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1299/2588] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:23 2022] DELETE /identity/v3/users/72703673ad964c42967fe239bad421ee => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:24.330635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.331162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.331562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:24.358915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:24.360406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.439615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1290/2589] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:24.451679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:24.453198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:24.523594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa693-8dd1-4609-931d-8170f4c6a5eb admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:24.527402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:24.528064 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:24.528353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:24.528633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:24.538517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:24.540055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:24.551464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1300/2590] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:25.089516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:25.091320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:25.105836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041a4af-0bce-4da1-82bf-8f5cd3989438 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:25.112394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:25.112984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:25.113269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:25.113554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] PATH_INFO: `/v3/projects/85a7a08436a94038af6a7606a87ad96b` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.114888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a7a08436a94038af6a7606a87ad96b)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:25.116872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:25.245995 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a7a08436a94038af6a7606a87ad96b', 'request_id': 'req-9a87031b-fc64-44ac-824a-e696c4c59b91'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.252345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a7a08436a94038af6a7606a87ad96b', 'request_id': 'req-9a87031b-fc64-44ac-824a-e696c4c59b91'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.334794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a7a08436a94038af6a7606a87ad96b', 'request_id': 'req-9a87031b-fc64-44ac-824a-e696c4c59b91'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.347057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] The token cache is being invalidate because project 85a7a08436a94038af6a7606a87ad96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:25.349805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.349805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.349805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1291/2591] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:25 2022] DELETE /identity/v3/projects/85a7a08436a94038af6a7606a87ad96b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:25.354462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:25.364991 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d190450-6863-43ed-81e2-019412a41e49 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:25.398782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:25.426171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:25.430259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:25.430259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:25.430259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] PATH_INFO: `/v3/users/6b083a07f4614623976b479a722fd207` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:25.430259 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b083a07f4614623976b479a722fd207)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:25.430651 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:25.669515 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b083a07f4614623976b479a722fd207'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.752559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b083a07f4614623976b479a722fd207'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.761088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b083a07f4614623976b479a722fd207'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.774048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b083a07f4614623976b479a722fd207'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.774785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-cf1276df-0d7b-40e5-8d91-15a69fbd2fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b083a07f4614623976b479a722fd207'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:25.781654 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1301/2592] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:25 2022] DELETE /identity/v3/users/6b083a07f4614623976b479a722fd207 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:26.269341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:26.270888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:26.282164 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:26.286131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87031b-fc64-44ac-824a-e696c4c59b91 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:26.371307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:26.372195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.372745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.373227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] PATH_INFO: `/v3/projects/de7bd67abc8b4de3bb223db063b7e63e` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.375118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de7bd67abc8b4de3bb223db063b7e63e)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:26.377299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:26.496930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de7bd67abc8b4de3bb223db063b7e63e', 'request_id': 'req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:26.503095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de7bd67abc8b4de3bb223db063b7e63e', 'request_id': 'req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:26.595785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de7bd67abc8b4de3bb223db063b7e63e', 'request_id': 'req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:26.607609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] The token cache is being invalidate because project de7bd67abc8b4de3bb223db063b7e63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:26.607911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:26.608554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b3002c0f-51ef-4f6a-9be7-bd281d418ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:26.610097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1292/2593] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:26 2022] DELETE /identity/v3/projects/de7bd67abc8b4de3bb223db063b7e63e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:26.615349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.615844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.615964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.636713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:26.637921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:26.699113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1302/2594] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:26.707593 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.707878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.708193 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.731221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:26.732527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:26.741720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1293/2595] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:26.746301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:26.756950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141a13b-7296-4648-8c3f-2d00f1921d79 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:26.762608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:26.763233 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.763505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.763779 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.764506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:26.764808 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:26.766433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:26.781776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1303/2596] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:34:26.784969 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:26.795233 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69d997-4818-4225-8f49-6160e9b8f20d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:26.800047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:26.801265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.801265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.801265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.802140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:26.804096 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:26.923952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1294/2597] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:26 2022] POST /identity/v3/projects => generated 361 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:26.928444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:26.938796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d9880-18d2-4d24-95f7-80f458f0ed43 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:26.982958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:26.997959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:26.998581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:26.998857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:26.999184 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:26.999824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:27.001840 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:27.099404 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1304/2598] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:26 2022] POST /identity/v3/users => generated 281 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:27.107501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:27.118766 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f6481-9996-46fd-ba13-38320b7ac968 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:27.124906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:27.125684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.125966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.126384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.127052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:27.128949 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:27.140507 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1295/2599] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:27.144647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:27.155818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc836d32-c051-4098-87e9-abb48ffe07b9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:27.160946 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8216cb-8a9a-4ec9-b236-681d3318b365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:27.161545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8216cb-8a9a-4ec9-b236-681d3318b365 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.161879 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8216cb-8a9a-4ec9-b236-681d3318b365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.162122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8216cb-8a9a-4ec9-b236-681d3318b365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.162788 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8216cb-8a9a-4ec9-b236-681d3318b365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:27.164464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8216cb-8a9a-4ec9-b236-681d3318b365 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:27.176040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1305/2600] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:27.180673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:27.191288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8babd-ec31-41d4-890f-7183cd939e79 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:27.196123 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:27.202144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.202144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.202144 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] PATH_INFO: `/v3/projects/aa25f47c230d4fb58fa9a4bdeee176a3/users/d50f99fc926f40ff80910e60ae3c4e34/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.214223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa25f47c230d4fb58fa9a4bdeee176a3, user_id=d50f99fc926f40ff80910e60ae3c4e34, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:27.216960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:27.318366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1296/2601] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:27 2022] PUT /identity/v3/projects/aa25f47c230d4fb58fa9a4bdeee176a3/users/d50f99fc926f40ff80910e60ae3c4e34/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:27.331389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.331684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.332020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.367680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] MFA Rules not processed for user `d50f99fc926f40ff80910e60ae3c4e34`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:27.369422 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:27.448292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1306/2602] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:27.460576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:27.463665 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:27.511824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:27.545112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633898-7849-4440-a85c-1c058e5f01c0 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:27.548894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:27.549513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:27.549794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:27.550066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:27.559845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:27.561386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:27.574553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1297/2603] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:37.444147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:38.489229 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=12 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:42.221498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:42.223634 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:42.433426 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75757221-20a2-4200-8325-c7e4f105cfad None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:42.440420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:42.440981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:42.441146 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:42.441453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:42.443446 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:42.462617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:42.464685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:42.482839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1307/2604] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:49.819937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:49.821373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:49.935052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc54da0-050b-454d-9af2-b8b540be8297 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:49.940145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:49.945446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:49.945446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:49.945446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:49.945446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:50.041521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:50.042994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:50.068388 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1298/2605] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:34:52.471838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=63 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:53.798880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:53.809609 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf356dbe-5e79-444a-914e-0ea1bf4b9632 service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:53.816494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:53.817047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:53.817324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:53.817596 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:53.819366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:53.833747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:53.835857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:53.848270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1308/2606] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:54.033049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:54.034784 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:54.050019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b82baf-62af-4835-9e49-1d87a35d76f7 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:54.055453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:54.056213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.057290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.057568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] PATH_INFO: `/v3/users/0e3f8c861a9c49119fc531e2a159ec9d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.058786 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3f8c861a9c49119fc531e2a159ec9d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:54.060827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:54.285585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:54.287290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3f8c861a9c49119fc531e2a159ec9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.287999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3f8c861a9c49119fc531e2a159ec9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.300939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3f8c861a9c49119fc531e2a159ec9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.301727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3f8c861a9c49119fc531e2a159ec9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.309547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-63eb1778-ee44-4616-84e4-2aee42350f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3f8c861a9c49119fc531e2a159ec9d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.420927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1299/2607] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:34:54 2022] DELETE /identity/v3/users/0e3f8c861a9c49119fc531e2a159ec9d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:54.744645 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:54.758914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d9e37-9c40-425b-a335-d5f6180fbba4 service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:54.847363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:54.848407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:54.848407 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:54.848699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] PATH_INFO: `/v3/projects/5bc75af3ba6c442889ce29351414e110` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:54.849778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc75af3ba6c442889ce29351414e110)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:54.851807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:54.989676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc75af3ba6c442889ce29351414e110', 'request_id': 'req-f928d52c-5317-4085-ad4a-3c93a042dfa1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:54.996292 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc75af3ba6c442889ce29351414e110', 'request_id': 'req-f928d52c-5317-4085-ad4a-3c93a042dfa1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.070768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc75af3ba6c442889ce29351414e110', 'request_id': 'req-f928d52c-5317-4085-ad4a-3c93a042dfa1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.085339 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] The token cache is being invalidate because project 5bc75af3ba6c442889ce29351414e110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:34:55.085710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.086296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-f928d52c-5317-4085-ad4a-3c93a042dfa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:34:55.088494 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1309/2608] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:34:54 2022] DELETE /identity/v3/projects/5bc75af3ba6c442889ce29351414e110 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:55.093541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.094361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.094361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.156703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:55.157930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.224040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1300/2609] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:55.233033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.233243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.233468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.255227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:55.256504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:55.267026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1310/2610] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:55.272018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.282254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01cba0-fd8c-43d5-b2bd-88922742e23f None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.287366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.288025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.288316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.288591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.289304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:34:55.289612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.291290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.304833 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1301/2611] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:34:55.310456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.323178 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfabb5-8e0e-4e43-a837-21d7f6c98b1b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.330156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.331015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.331413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.331700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.332865 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.336217 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.452886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1311/2612] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/projects => generated 401 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:55.455612 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.466809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0133d-5c1c-4c7c-95dc-133655760465 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.528558 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.529171 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.529456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.529736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.530379 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.532508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.630143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1302/2613] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/users => generated 301 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:55.634656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.656679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eebee-825a-4a11-97ae-b68abcf8d338 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.661403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.662015 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.662297 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.662576 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.663226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.664872 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.677488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1312/2614] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:55.682395 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.692649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217be349-5835-4eed-a1fa-13c42951e15a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.697416 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d1628-b368-43a5-b160-a0d94c0a2aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.698007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d1628-b368-43a5-b160-a0d94c0a2aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.698284 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d1628-b368-43a5-b160-a0d94c0a2aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.698555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d1628-b368-43a5-b160-a0d94c0a2aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.699195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d1628-b368-43a5-b160-a0d94c0a2aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.700819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d1628-b368-43a5-b160-a0d94c0a2aa9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.703093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:55.713047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1303/2615] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:34:55.723359 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:55.735516 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1813129-fe96-47f1-97cf-b2c2cdf1e78f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:55.740362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:55.741030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.741305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.741579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] PATH_INFO: `/v3/projects/67b686dc1a254a4db31763535efa5f43/users/23adf8d7c8a94044b9d006666da2a597/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.758031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b686dc1a254a4db31763535efa5f43, user_id=23adf8d7c8a94044b9d006666da2a597, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:55.760799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:55.884029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1313/2616] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:55 2022] PUT /identity/v3/projects/67b686dc1a254a4db31763535efa5f43/users/23adf8d7c8a94044b9d006666da2a597/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:34:55.896058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:55.896343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:55.896619 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:55.928047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] MFA Rules not processed for user `23adf8d7c8a94044b9d006666da2a597`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:55.929352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.019396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1304/2617] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:34:56.025224 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.049307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:56.110521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d03c9-e997-4e9b-8ee7-7d0c54bf159c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.116848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.118041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.118474 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.118986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.120190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.124157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.248603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1314/2618] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/projects => generated 401 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:56.253106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.264602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6190ff-a558-4d34-8c91-c96ac505d021 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.328536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.329156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.329442 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.329720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.330357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.332371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.437693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1305/2619] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/users => generated 307 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:34:56.441900 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.453666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e2540b-6766-40dd-92c1-cf870b4a0e0c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.459545 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.460188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.460472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.460749 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.461495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.463720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.477610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1315/2620] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:56.482497 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.493568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318e525-2ace-4b25-a214-e2aa89c4bd44 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.498460 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.499083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.499267 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.499572 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] PATH_INFO: `/v3/projects/29c790fcee07477fb2729ea9f12b7a1e/users/93d843359f244c31a4e0f4e0402639ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.517576 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c790fcee07477fb2729ea9f12b7a1e, user_id=93d843359f244c31a4e0f4e0402639ae, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.520384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.672543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1306/2621] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:56 2022] PUT /identity/v3/projects/29c790fcee07477fb2729ea9f12b7a1e/users/93d843359f244c31a4e0f4e0402639ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:56.676893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.688841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66553c2-6e56-4a05-97d2-de669ad8ac50 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.752953 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed874322-d86a-4ca4-b9e2-01e747ce98ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.753655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed874322-d86a-4ca4-b9e2-01e747ce98ad admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.753940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed874322-d86a-4ca4-b9e2-01e747ce98ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.754243 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed874322-d86a-4ca4-b9e2-01e747ce98ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.754930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed874322-d86a-4ca4-b9e2-01e747ce98ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.756640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed874322-d86a-4ca4-b9e2-01e747ce98ad admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.768270 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1316/2622] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:34:56.787928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.787928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d005760-46d7-476e-a929-70c839e1c270 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:56.789679 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:56.790385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.790694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.791010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] PATH_INFO: `/v3/projects/29c790fcee07477fb2729ea9f12b7a1e/users/93d843359f244c31a4e0f4e0402639ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.793356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c790fcee07477fb2729ea9f12b7a1e, user_id=93d843359f244c31a4e0f4e0402639ae, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:56.796014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:56.808499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1307/2623] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:34:56 2022] PUT /identity/v3/projects/29c790fcee07477fb2729ea9f12b7a1e/users/93d843359f244c31a4e0f4e0402639ae/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:34:56.819629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:56.819955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:56.820241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:56.854818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] MFA Rules not processed for user `93d843359f244c31a4e0f4e0402639ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:34:56.856168 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.931440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1317/2624] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:34:56 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:34:56.943099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:56.945212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:56.995325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:57.034277 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32e7d6-8cf1-42c1-ba7c-a82ce221612c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.038112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.038733 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.039009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.039288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.049724 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.051712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.062521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1308/2625] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:56 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:34:57.380777 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:34:57.382277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:34:57.409041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b35a9-7518-40d1-873b-21fcf952080c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:34:57.413664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f197eb3-8c46-4c17-9a5b-36382fdb0f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:34:57.416508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f197eb3-8c46-4c17-9a5b-36382fdb0f3e service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:34:57.416508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f197eb3-8c46-4c17-9a5b-36382fdb0f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:34:57.416508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f197eb3-8c46-4c17-9a5b-36382fdb0f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:34:57.422556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:34:57.494741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f197eb3-8c46-4c17-9a5b-36382fdb0f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:34:57.496240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f197eb3-8c46-4c17-9a5b-36382fdb0f3e service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:34:57.506985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1318/2626] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:34:57 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:02.971860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.040583 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11a32-1dbd-4081-88e7-6330535818e4 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.047845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.048517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.048802 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.049079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] PATH_INFO: `/v3/users/23adf8d7c8a94044b9d006666da2a597` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.050252 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=23adf8d7c8a94044b9d006666da2a597)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.052236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.310850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23adf8d7c8a94044b9d006666da2a597'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.311516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23adf8d7c8a94044b9d006666da2a597'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.324456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23adf8d7c8a94044b9d006666da2a597'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.325192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23adf8d7c8a94044b9d006666da2a597'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.335090 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23adf8d7c8a94044b9d006666da2a597'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:03.476487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1309/2627] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:02 2022] DELETE /identity/v3/users/23adf8d7c8a94044b9d006666da2a597 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:03.483483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.484075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.484221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.507380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:03.508690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.574349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1319/2628] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:03.584254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:03.585776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:03.640282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:03.653812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad298-c3e7-4ddf-9ae3-c8b572b152ca admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:03.657996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:03.658941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:03.658941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:03.659227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:03.669757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:03.671276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:03.682206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1310/2629] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:04.188363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.190385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:04.203286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea77ac-f816-43c4-8816-3e1b77eebd5f None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.208246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.208871 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.209152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.209428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] PATH_INFO: `/v3/projects/67b686dc1a254a4db31763535efa5f43` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.210564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b686dc1a254a4db31763535efa5f43)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.212595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.350586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b686dc1a254a4db31763535efa5f43', 'request_id': 'req-c3828d23-a259-4759-b1ae-7b2924e6f49a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.356478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b686dc1a254a4db31763535efa5f43', 'request_id': 'req-c3828d23-a259-4759-b1ae-7b2924e6f49a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.436932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b686dc1a254a4db31763535efa5f43', 'request_id': 'req-c3828d23-a259-4759-b1ae-7b2924e6f49a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.457020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] The token cache is being invalidate because project 67b686dc1a254a4db31763535efa5f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:04.457355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.458435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.461065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1320/2630] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:04 2022] DELETE /identity/v3/projects/67b686dc1a254a4db31763535efa5f43 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:04.467521 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:04.484466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6874c-88c8-4305-8970-2832b4a6aac6 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:04.546857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:04.547503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:04.547816 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:04.548268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] PATH_INFO: `/v3/users/93d843359f244c31a4e0f4e0402639ae` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:04.549486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d843359f244c31a4e0f4e0402639ae)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:04.551429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:04.853555 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d843359f244c31a4e0f4e0402639ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.853910 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d843359f244c31a4e0f4e0402639ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.870085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d843359f244c31a4e0f4e0402639ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.871639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d843359f244c31a4e0f4e0402639ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.878630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8b40a5a6-0259-43e9-b348-4f36d4e626b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d843359f244c31a4e0f4e0402639ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:04.995727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1311/2631] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:04 2022] DELETE /identity/v3/users/93d843359f244c31a4e0f4e0402639ae => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:05.552028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:05.554108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:05.570853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828d23-a259-4759-b1ae-7b2924e6f49a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:05.605320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:05.638395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:05.639079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.639372 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.643848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] PATH_INFO: `/v3/projects/29c790fcee07477fb2729ea9f12b7a1e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.643848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c790fcee07477fb2729ea9f12b7a1e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:05.643848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:05.800299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c790fcee07477fb2729ea9f12b7a1e', 'request_id': 'req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.805985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c790fcee07477fb2729ea9f12b7a1e', 'request_id': 'req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.934740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c790fcee07477fb2729ea9f12b7a1e', 'request_id': 'req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.947140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] The token cache is being invalidate because project 29c790fcee07477fb2729ea9f12b7a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:05.947432 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.948696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0f4e4834-a735-4a2d-98e4-e9641e46f4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:05.950226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1321/2632] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:05 2022] DELETE /identity/v3/projects/29c790fcee07477fb2729ea9f12b7a1e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:05.955662 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:05.956359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:05.956541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:05.978714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:05.979984 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:06.045735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1312/2633] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:06.054396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.054685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.055084 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.078244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:06.080945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:06.088536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1322/2634] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:06.093717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.101428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:06.106673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185accd1-bee5-4688-a30c-d53752e9466d None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.111549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.112220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.112484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.112757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.113585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:06.113892 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.115509 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.127798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1313/2635] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:06.132315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.142577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5898f52-97e1-4d02-b820-1cc14b541a7a None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.147758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.148734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.148734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.149163 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.149761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.152185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.296479 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1323/2636] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:06 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:06.301033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.312414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b360c46-e674-4eea-9a16-bd124368e2a7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.371561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.371830 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.372166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.372437 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.373059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.375003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.486887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1314/2637] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:06 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:06.491294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.502350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134acbe-df14-4b98-8f5e-34e6de512470 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.507386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.508056 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.508337 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.508608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.509249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.511519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.522269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1324/2638] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:06.526547 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.537477 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0c30f-dc5d-4960-af21-32d161d4d002 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.545560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceec32-be13-4835-85fd-c6857741e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.545560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceec32-be13-4835-85fd-c6857741e639 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.545560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceec32-be13-4835-85fd-c6857741e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.545560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceec32-be13-4835-85fd-c6857741e639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.545908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceec32-be13-4835-85fd-c6857741e639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.546069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceec32-be13-4835-85fd-c6857741e639 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.558430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1315/2639] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:06.563506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.575134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40dfe0-35e3-4221-8a64-9fc409bdf73b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.579928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.580607 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.580885 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.581166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] PATH_INFO: `/v3/projects/9ba76ba44eb94c93a9e90974151a04e5/users/459e0a285003437e99bd106d9b3fb822/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.597690 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba76ba44eb94c93a9e90974151a04e5, user_id=459e0a285003437e99bd106d9b3fb822, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.600478 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:06.712258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1325/2640] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:06 2022] PUT /identity/v3/projects/9ba76ba44eb94c93a9e90974151a04e5/users/459e0a285003437e99bd106d9b3fb822/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:06.725057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.725410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.725750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.760394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] MFA Rules not processed for user `459e0a285003437e99bd106d9b3fb822`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:06.762757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:06.857675 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1316/2641] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:06.862569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:06.926174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:06.930290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c35dc-ef65-408b-bcaa-99d9254700c4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:06.935652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:06.936395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:06.936584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:06.936863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:06.938058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:06.940002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.069298 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1326/2642] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:06 2022] POST /identity/v3/projects => generated 381 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:07.073645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.086133 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdbbe2-7410-4fa2-939e-0ba3c9e90b2c None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:07.156709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:07.157327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.157611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.157886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.158526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:07.160566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.169568 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:07.286856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1317/2643] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:07 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:07.291088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.302328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e0847-7a5a-4e4f-a0ba-0dc0f7043777 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:07.306987 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:07.307713 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.308028 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.308315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.308967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:07.310736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.326241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1327/2644] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:07.327981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.341412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867e83b-4015-45f6-8874-8b2c67b3f80a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:07.347560 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:07.348565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.348942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.349223 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] PATH_INFO: `/v3/projects/1a142490b1b1490f83285119d8918b11/users/109bbbd47d074e928b332fb7ea7b5281/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.367752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a142490b1b1490f83285119d8918b11, user_id=109bbbd47d074e928b332fb7ea7b5281, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:07.370641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.471020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1318/2645] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:07 2022] PUT /identity/v3/projects/1a142490b1b1490f83285119d8918b11/users/109bbbd47d074e928b332fb7ea7b5281/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:07.475220 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.486419 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ae3c9-b6f1-452d-9a88-ff01d815cb24 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:07.546540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bed61-a2d3-492d-ab89-c248b6cd947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:07.547173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bed61-a2d3-492d-ab89-c248b6cd947e admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.547452 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bed61-a2d3-492d-ab89-c248b6cd947e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.547728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bed61-a2d3-492d-ab89-c248b6cd947e admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.548405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bed61-a2d3-492d-ab89-c248b6cd947e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:07.550042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bed61-a2d3-492d-ab89-c248b6cd947e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.560762 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1328/2646] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:07.565269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.585504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5b152-61da-455f-bb83-9f05cbf2f61b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:07.585504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:07.585504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.585504 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.586010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] PATH_INFO: `/v3/projects/1a142490b1b1490f83285119d8918b11/users/109bbbd47d074e928b332fb7ea7b5281/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.586010 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a142490b1b1490f83285119d8918b11, user_id=109bbbd47d074e928b332fb7ea7b5281, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:07.587657 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.599759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1319/2647] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:07 2022] PUT /identity/v3/projects/1a142490b1b1490f83285119d8918b11/users/109bbbd47d074e928b332fb7ea7b5281/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:07.610223 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.610814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.610814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.642434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] MFA Rules not processed for user `109bbbd47d074e928b332fb7ea7b5281`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:07.643744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:07.722511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1329/2648] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:07.735716 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:07.737603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:07.800811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596fa1c-50ff-41f9-a10d-219de0eb41ad admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:07.804788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:07.805405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:07.805701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:07.805972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:07.816118 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:07.817783 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:07.828475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1320/2649] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:08.021002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.031023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c466f-4e65-4eb2-aac8-755b7be36024 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.034966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36dc552-928d-48d9-aab0-53a58bc463d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.035573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc552-928d-48d9-aab0-53a58bc463d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.035850 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc552-928d-48d9-aab0-53a58bc463d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.036156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36dc552-928d-48d9-aab0-53a58bc463d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.087603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:08.097403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dc552-928d-48d9-aab0-53a58bc463d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.106890 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36dc552-928d-48d9-aab0-53a58bc463d5 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.107794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1330/2650] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:08.645649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:08.711110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eeabf8-819a-4966-afd7-bc237c3ea59e service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:08.716445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:08.717147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:08.717430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:08.717694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] PATH_INFO: `/v3/users/459e0a285003437e99bd106d9b3fb822` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:08.718834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] RBAC: Authorizing `identity:delete_user(user_id=459e0a285003437e99bd106d9b3fb822)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:08.721101 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:08.958212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459e0a285003437e99bd106d9b3fb822'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.958212 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459e0a285003437e99bd106d9b3fb822'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.966812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:08.973112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459e0a285003437e99bd106d9b3fb822'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.973880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459e0a285003437e99bd106d9b3fb822'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:08.981450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459e0a285003437e99bd106d9b3fb822'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:09.112242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1321/2651] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:08 2022] DELETE /identity/v3/users/459e0a285003437e99bd106d9b3fb822 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:09.116828 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.117327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.117423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.138086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:09.139218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.199305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1331/2652] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:09.209468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.210878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.271272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f3169-a664-4ed9-839a-0a95add9349a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.275073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.275688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.276002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.276887 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.286160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.287607 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.297965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1322/2653] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:09.741838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:09.743325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:09.756917 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6640d2d-1b10-4003-80dd-35c9e066950b None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:09.763262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:09.763262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:09.763262 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:09.763928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] PATH_INFO: `/v3/projects/9ba76ba44eb94c93a9e90974151a04e5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:09.765180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba76ba44eb94c93a9e90974151a04e5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:09.767280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:09.924377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba76ba44eb94c93a9e90974151a04e5', 'request_id': 'req-2aa70ef0-f97f-460a-a380-54699ae2cce1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:09.930106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba76ba44eb94c93a9e90974151a04e5', 'request_id': 'req-2aa70ef0-f97f-460a-a380-54699ae2cce1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.060854 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba76ba44eb94c93a9e90974151a04e5', 'request_id': 'req-2aa70ef0-f97f-460a-a380-54699ae2cce1'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.073592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] The token cache is being invalidate because project 9ba76ba44eb94c93a9e90974151a04e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:10.073889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.074555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.076112 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1332/2654] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:09 2022] DELETE /identity/v3/projects/9ba76ba44eb94c93a9e90974151a04e5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:10.080573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:10.092917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a99b6-2c41-43a4-9c57-ccf704699b44 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:10.154711 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:10.155355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:10.155635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:10.155936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] PATH_INFO: `/v3/users/109bbbd47d074e928b332fb7ea7b5281` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:10.157089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=109bbbd47d074e928b332fb7ea7b5281)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:10.159043 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:10.393360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109bbbd47d074e928b332fb7ea7b5281'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.394107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109bbbd47d074e928b332fb7ea7b5281'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.407332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109bbbd47d074e928b332fb7ea7b5281'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.408275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109bbbd47d074e928b332fb7ea7b5281'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.416253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-61ef4862-145e-498c-8303-6a18a7b0ed6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109bbbd47d074e928b332fb7ea7b5281'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:10.421386 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:10.517749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1323/2655] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:10 2022] DELETE /identity/v3/users/109bbbd47d074e928b332fb7ea7b5281 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:11.023261 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:11.024739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:11.040071 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa70ef0-f97f-460a-a380-54699ae2cce1 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:11.111134 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:11.111785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:11.112118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:11.112393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] PATH_INFO: `/v3/projects/1a142490b1b1490f83285119d8918b11` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:11.113579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a142490b1b1490f83285119d8918b11)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:11.115505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:11.139211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:11.313026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a142490b1b1490f83285119d8918b11', 'request_id': 'req-c762b315-1191-4ed2-a670-17efd1750ab6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:11.318804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a142490b1b1490f83285119d8918b11', 'request_id': 'req-c762b315-1191-4ed2-a670-17efd1750ab6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:11.430936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a142490b1b1490f83285119d8918b11', 'request_id': 'req-c762b315-1191-4ed2-a670-17efd1750ab6'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:11.442803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] The token cache is being invalidate because project 1a142490b1b1490f83285119d8918b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:11.443124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:11.443753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c762b315-1191-4ed2-a670-17efd1750ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:11.445289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1333/2656] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:11 2022] DELETE /identity/v3/projects/1a142490b1b1490f83285119d8918b11 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:11.450878 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:11.451398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:11.451513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:11.472659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:11.473919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:11.533121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1324/2657] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:11.541349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:11.541672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:11.541944 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:11.562232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:11.563719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:11.571768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1334/2658] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:11.581315 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:11.591231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50690a37-d431-42f8-948d-852fb1ce2a47 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:11.596081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:11.596695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:11.596975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:11.597247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:11.597896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:11.598203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:11.600059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:11.612720 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1325/2659] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:11.616922 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:11.626457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c588d4-a8ea-487e-8e19-8d7536ea630c None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:11.631580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:11.632237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:11.632524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:11.632789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:11.633412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:11.635324 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:11.781847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1335/2660] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:11 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:11.786559 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:11.797336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac884c-cc5c-4dd5-aeb0-3c867cc15276 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:11.864001 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:11.864685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:11.864970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:11.865247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:11.865904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:11.867844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.129734 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1326/2661] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:11 2022] POST /identity/v3/users => generated 286 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:12.134198 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.145472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6340c29-095b-4f1e-8557-357add453681 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.151461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.152123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.152401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.152673 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.153327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.155054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.166152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1336/2662] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:12.170845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.185553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f14b3-e739-4384-9fb1-07e46dad52a2 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.190502 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.191116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.191399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.191668 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.192365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.193968 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.206031 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1327/2663] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:12.210280 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.222307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c03b27-0dbf-44fe-922d-9431b77c4174 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.225974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.227285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9590-0214-4748-a4ce-208e5b12492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.227642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:12.228065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9590-0214-4748-a4ce-208e5b12492e admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.228350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9590-0214-4748-a4ce-208e5b12492e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.228626 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9590-0214-4748-a4ce-208e5b12492e admin admin] PATH_INFO: `/v3/projects/11169e8b127044a09efa5640e8fc54f7/users/95a453adcd8d486a952f1dd79d642ff3/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.243343 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cecc840-6d74-451b-8bf0-288f3d1f203d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.245699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9590-0214-4748-a4ce-208e5b12492e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11169e8b127044a09efa5640e8fc54f7, user_id=95a453adcd8d486a952f1dd79d642ff3, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.249004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9590-0214-4748-a4ce-208e5b12492e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.249359 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.250005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.250361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.250594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] PATH_INFO: `/v3/users/d50f99fc926f40ff80910e60ae3c4e34` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.251747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50f99fc926f40ff80910e60ae3c4e34)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.254820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.456720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1337/2664] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:12 2022] PUT /identity/v3/projects/11169e8b127044a09efa5640e8fc54f7/users/95a453adcd8d486a952f1dd79d642ff3/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:12.471823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.472159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.472440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.504948 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] MFA Rules not processed for user `95a453adcd8d486a952f1dd79d642ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:12.506283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:12.581536 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1338/2665] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:12.594640 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.596137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:12.610715 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50f99fc926f40ff80910e60ae3c4e34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:12.611957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50f99fc926f40ff80910e60ae3c4e34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:12.626499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:12.632610 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50f99fc926f40ff80910e60ae3c4e34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:12.633445 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50f99fc926f40ff80910e60ae3c4e34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:12.642800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50f99fc926f40ff80910e60ae3c4e34'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:12.644860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:12.671246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532de16-68e2-40e5-9de8-dc69398619b7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.675365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.676052 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.676383 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.676686 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.678556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:12.695042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.696753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625efaf-df5a-4c2c-a7b2-663c725a30ef service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.707812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1339/2666] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:12.732540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1328/2667] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:12 2022] DELETE /identity/v3/users/d50f99fc926f40ff80910e60ae3c4e34 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:12.739247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b7a64-833f-49a1-9822-47681564df34 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.739540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b7a64-833f-49a1-9822-47681564df34 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.739931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b7a64-833f-49a1-9822-47681564df34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.761716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-251b7a64-833f-49a1-9822-47681564df34 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:12.763533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-251b7a64-833f-49a1-9822-47681564df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:12.852121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1340/2668] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:12.866719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:12.869534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:12.945129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45507202-93cb-4957-bf25-2d5e03c90310 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:12.963304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870b5a0-15c2-4996-a0d9-0d48144e0180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:12.963304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b5a0-15c2-4996-a0d9-0d48144e0180 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:12.963304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b5a0-15c2-4996-a0d9-0d48144e0180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:12.963304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b5a0-15c2-4996-a0d9-0d48144e0180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:12.963944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870b5a0-15c2-4996-a0d9-0d48144e0180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:12.966129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870b5a0-15c2-4996-a0d9-0d48144e0180 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:12.978064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1329/2669] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:13.534913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b7a64-833f-49a1-9822-47681564df34 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:13.536525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-251b7a64-833f-49a1-9822-47681564df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:13.552806 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b7a64-833f-49a1-9822-47681564df34 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:13.561616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:13.562792 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:13.563301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:13.563834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] PATH_INFO: `/v3/projects/aa25f47c230d4fb58fa9a4bdeee176a3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:13.566103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa25f47c230d4fb58fa9a4bdeee176a3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:13.570236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:13.719363 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa25f47c230d4fb58fa9a4bdeee176a3', 'request_id': 'req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:13.725797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa25f47c230d4fb58fa9a4bdeee176a3', 'request_id': 'req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:13.840232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa25f47c230d4fb58fa9a4bdeee176a3', 'request_id': 'req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:13.855305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] The token cache is being invalidate because project aa25f47c230d4fb58fa9a4bdeee176a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:13.855603 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:13.856331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d68fc55f-04a2-40fb-80de-6cd82e00c0cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:13.858000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1341/2670] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:13 2022] DELETE /identity/v3/projects/aa25f47c230d4fb58fa9a4bdeee176a3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:13.863367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:13.863946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:13.863946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:13.886071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:13.888149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:13.964778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1330/2671] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:13.975468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:13.979204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:13.979204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:13.997398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:13.998624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:14.007470 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1342/2672] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:14.012185 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.022917 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027588-08ea-40a0-847b-0b8a2cdc8fd4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.027676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.028817 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.029104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.029376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.030052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:14.030351 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.032126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.045209 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1331/2673] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:14.050155 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.060005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b595d-a020-44f3-9e9b-10c13ab66826 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.070623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.070623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.070623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.070623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.071226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.071226 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.199030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1343/2674] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:14 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:14.204100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.214420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce08fd-787f-4ddb-b5bd-c130ec80f9a7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.272452 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.273072 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.273356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.273632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.274281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.276314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.289785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:14.402728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1332/2675] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:14 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:14.416097 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.427518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c449307-a2bb-4fa5-8484-8dbf5ae027f2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.434705 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.435328 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.435608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.435901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.436579 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.438447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.451246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1344/2676] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:14.455630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.467981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552bb54-05eb-46fb-ba49-aa410334d78b admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.475680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a58788-192d-4cb3-bd0e-c019f511ebfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.475680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a58788-192d-4cb3-bd0e-c019f511ebfa admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.475680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a58788-192d-4cb3-bd0e-c019f511ebfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.475680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a58788-192d-4cb3-bd0e-c019f511ebfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.476278 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a58788-192d-4cb3-bd0e-c019f511ebfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.477135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a58788-192d-4cb3-bd0e-c019f511ebfa admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.488906 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1333/2677] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:14.493219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.504481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e83d7-3f18-4ca4-9743-2c7993ad57ee admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.511094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.511776 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.512118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.512990 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] PATH_INFO: `/v3/projects/abd71d237e054c94aba2af45655b6632/users/cdef060ee45e4bf58cc8e81fb7f61f63/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.529901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd71d237e054c94aba2af45655b6632, user_id=cdef060ee45e4bf58cc8e81fb7f61f63, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.532815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.535172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:14.683196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1345/2678] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:14 2022] PUT /identity/v3/projects/abd71d237e054c94aba2af45655b6632/users/cdef060ee45e4bf58cc8e81fb7f61f63/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:14.697400 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.697701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.697979 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.731374 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] MFA Rules not processed for user `cdef060ee45e4bf58cc8e81fb7f61f63`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:14.732444 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:14.809767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1334/2679] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:14.828118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.830194 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:14.830726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:14.833523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:14.922480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc101c-daff-4306-a85f-736d46f4d9c1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.926221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38702ba-c731-4e47-aa10-5f38e306a9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.926847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38702ba-c731-4e47-aa10-5f38e306a9ba service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.927137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38702ba-c731-4e47-aa10-5f38e306a9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.927396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38702ba-c731-4e47-aa10-5f38e306a9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.934712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5d1f1-1e34-4074-be6a-6bc7690a1aa3 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:14.940701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38702ba-c731-4e47-aa10-5f38e306a9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.942189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38702ba-c731-4e47-aa10-5f38e306a9ba service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.942352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:14.943327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:14.943787 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:14.944304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] PATH_INFO: `/v3/users/feca3de18907429694445faecedd083a` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:14.946309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] RBAC: Authorizing `identity:delete_user(user_id=feca3de18907429694445faecedd083a)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:14.950402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:14.953652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1335/2680] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:15.221029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feca3de18907429694445faecedd083a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:15.356929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feca3de18907429694445faecedd083a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:15.364921 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feca3de18907429694445faecedd083a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:15.378390 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feca3de18907429694445faecedd083a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:15.379281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feca3de18907429694445faecedd083a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:15.385599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1346/2681] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:14 2022] DELETE /identity/v3/users/feca3de18907429694445faecedd083a => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:15.391827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:15.392214 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:15.392466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:15.412517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:15.413736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:15.479160 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1336/2682] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:15.491296 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:15.493009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:15.553161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dcd3f-8c91-4314-a09e-5aead151194d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:15.556855 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:15.557469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:15.557754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:15.558129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:15.567581 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:15.569126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:15.578902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1347/2683] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:16.133326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:16.134867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:16.153744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84566fa-33a7-4534-bd39-5d28677c8d07 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:16.159537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:16.160231 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:16.160515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:16.160787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] PATH_INFO: `/v3/projects/e3effb7cff314875b00b9f11b7a1e94a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:16.161992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3effb7cff314875b00b9f11b7a1e94a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:16.165865 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:16.176653 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:16.318618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3effb7cff314875b00b9f11b7a1e94a', 'request_id': 'req-0e9e26df-9161-48bc-8c51-e075eaee710c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.325468 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3effb7cff314875b00b9f11b7a1e94a', 'request_id': 'req-0e9e26df-9161-48bc-8c51-e075eaee710c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.391762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3effb7cff314875b00b9f11b7a1e94a', 'request_id': 'req-0e9e26df-9161-48bc-8c51-e075eaee710c'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.405765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] The token cache is being invalidate because project e3effb7cff314875b00b9f11b7a1e94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:16.406100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.406792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.411520 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1337/2684] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:16 2022] DELETE /identity/v3/projects/e3effb7cff314875b00b9f11b7a1e94a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:16.412955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:16.423483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c28edd-f97e-498e-9909-3195b4d08a36 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:16.435758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:16.487625 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:16.488315 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:16.488595 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:16.488867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] PATH_INFO: `/v3/users/d9cc506f8cfb4693bc036f051d2cc7e5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:16.490019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9cc506f8cfb4693bc036f051d2cc7e5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:16.492030 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:16.757558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9cc506f8cfb4693bc036f051d2cc7e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.843002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9cc506f8cfb4693bc036f051d2cc7e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.852231 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9cc506f8cfb4693bc036f051d2cc7e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.866653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9cc506f8cfb4693bc036f051d2cc7e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.867563 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-e7a73260-d8f6-4c29-9b37-dde9a354f985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9cc506f8cfb4693bc036f051d2cc7e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:16.874679 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1348/2685] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:16 2022] DELETE /identity/v3/users/d9cc506f8cfb4693bc036f051d2cc7e5 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:17.388138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:17.390791 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.406687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e26df-9161-48bc-8c51-e075eaee710c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.478189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.478854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.479138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.479417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] PATH_INFO: `/v3/projects/2d07ce1b0727438f84c3c1096eff345a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.480562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d07ce1b0727438f84c3c1096eff345a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.483219 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:17.626694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d07ce1b0727438f84c3c1096eff345a', 'request_id': 'req-56f037dd-3149-44d4-9cac-7f06f33e45d9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.634454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d07ce1b0727438f84c3c1096eff345a', 'request_id': 'req-56f037dd-3149-44d4-9cac-7f06f33e45d9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.734049 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d07ce1b0727438f84c3c1096eff345a', 'request_id': 'req-56f037dd-3149-44d4-9cac-7f06f33e45d9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.747228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] The token cache is being invalidate because project 2d07ce1b0727438f84c3c1096eff345a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:17.747595 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.751811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-56f037dd-3149-44d4-9cac-7f06f33e45d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:17.751811 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1338/2686] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:17 2022] DELETE /identity/v3/projects/2d07ce1b0727438f84c3c1096eff345a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:17.756142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.756459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.756760 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.780371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:17.786385 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.856044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1349/2687] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:17.864829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.865114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.865392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.889795 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:17.890963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:17.899243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1339/2688] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:17.904247 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:17.914568 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eec688-fe3a-4e02-9e25-b5ffdb1c20fc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.920428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.921073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.921356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.921630 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.922306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:17.922608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.924458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:17.938483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1350/2689] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:35:17.942453 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:17.953092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ffc6-a002-40e1-9a4e-6109846eaeb8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:17.961481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:17.961481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:17.961481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:17.961481 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:17.962026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:17.963184 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.129356 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:18.135901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1340/2690] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:17 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:18.149887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.152358 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb6bc6-2c9f-4832-9c7b-e189e35b22cc admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.173401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:18.242933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.244027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.244543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.245047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.246204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.250772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.386378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1351/2691] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:18 2022] POST /identity/v3/users => generated 294 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:18.391179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.401936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fd5e-f882-4f14-813d-6918a7c2b2dc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.406772 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.407417 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.407982 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.408265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.408927 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.410537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.421725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1341/2692] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:18.425986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.436782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cb33c-5b4b-498a-91f8-0138a852426c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.441397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb02804-a4b8-403d-aa28-7d70a3fa982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.442048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb02804-a4b8-403d-aa28-7d70a3fa982d admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.442327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb02804-a4b8-403d-aa28-7d70a3fa982d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.442599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb02804-a4b8-403d-aa28-7d70a3fa982d admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.443271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb02804-a4b8-403d-aa28-7d70a3fa982d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.444895 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb02804-a4b8-403d-aa28-7d70a3fa982d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.457166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1352/2693] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:18.461939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.473138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e80ab7-a0a7-4e6a-826d-f8400290512f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.478015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.478449 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.478730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.479006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] PATH_INFO: `/v3/projects/f8ea8b9fda9c47d194a46e7372305932/users/8a90830db73c49d1852256ca9d930495/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.494105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ea8b9fda9c47d194a46e7372305932, user_id=8a90830db73c49d1852256ca9d930495, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.497305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.623347 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1342/2694] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:18 2022] PUT /identity/v3/projects/f8ea8b9fda9c47d194a46e7372305932/users/8a90830db73c49d1852256ca9d930495/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:18.635448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.635737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.636054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.666195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] MFA Rules not processed for user `8a90830db73c49d1852256ca9d930495`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:18.667418 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:18.734573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1353/2695] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:18.739341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:18.812871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cb1af-bc58-4958-a2ed-46af7c3bb9b4 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:18.817924 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:18.818493 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:18.818849 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:18.819051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:18.819677 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:18.821714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:18.997272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1343/2696] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:18 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:18.998950 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.009936 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7382b-a4ff-4af9-bec3-bec798013db4 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.068933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.069554 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.069835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.070109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.070758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.072728 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.249646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1354/2697] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:18 2022] POST /identity/v3/users => generated 299 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:19.253875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.265287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d3e1-4c1a-40ba-bcb0-bfdf69183069 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.270896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.271514 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.271793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.272105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.272754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.274376 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.287473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1344/2698] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:19.291939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.302708 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55207987-08c7-47ec-8a57-7e6fffd13681 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.308267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.309252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.309521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.309920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] PATH_INFO: `/v3/projects/9fc31a323f104a38a9c8c7f99131fc0e/users/b8af2e22c549476e9bd0c4b8379b31e7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.326463 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc31a323f104a38a9c8c7f99131fc0e, user_id=b8af2e22c549476e9bd0c4b8379b31e7, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.329433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.332757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:19.465928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1355/2699] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:19 2022] PUT /identity/v3/projects/9fc31a323f104a38a9c8c7f99131fc0e/users/b8af2e22c549476e9bd0c4b8379b31e7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:19.470680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.480929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c21da4-9ac6-4fd1-849d-30ad26c34abc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.544682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c737e2-8cd7-4d3e-bf22-bcc534376d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.545227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c737e2-8cd7-4d3e-bf22-bcc534376d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.545510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c737e2-8cd7-4d3e-bf22-bcc534376d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.545996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c737e2-8cd7-4d3e-bf22-bcc534376d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.546430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c737e2-8cd7-4d3e-bf22-bcc534376d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.548068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c737e2-8cd7-4d3e-bf22-bcc534376d91 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.558803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1345/2700] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:19.563237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.575674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a772a0-84fd-406b-91f6-9114526c625e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.585027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.586308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.586618 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.586899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] PATH_INFO: `/v3/projects/9fc31a323f104a38a9c8c7f99131fc0e/users/b8af2e22c549476e9bd0c4b8379b31e7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.588951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc31a323f104a38a9c8c7f99131fc0e, user_id=b8af2e22c549476e9bd0c4b8379b31e7, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.592271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.608670 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1356/2701] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:19 2022] PUT /identity/v3/projects/9fc31a323f104a38a9c8c7f99131fc0e/users/b8af2e22c549476e9bd0c4b8379b31e7/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:19.619143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.619464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.619744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.649942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] MFA Rules not processed for user `b8af2e22c549476e9bd0c4b8379b31e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:19.651213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:19.676623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:19.737527 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1346/2702] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:19.740185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.741923 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:19.807444 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9a7b-0315-41aa-8bb1-1f1f63eb6ab5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.812190 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51421973-a424-467e-b4a7-e5d26475a2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.812807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51421973-a424-467e-b4a7-e5d26475a2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.813092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51421973-a424-467e-b4a7-e5d26475a2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.813366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51421973-a424-467e-b4a7-e5d26475a2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.823012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51421973-a424-467e-b4a7-e5d26475a2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.824534 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51421973-a424-467e-b4a7-e5d26475a2a4 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:19.836440 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1357/2703] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:19.867921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:19.937602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f862343-ab66-49bb-9389-b14f7e490ba8 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:19.942900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:19.943586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:19.943864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:19.944179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] PATH_INFO: `/v3/users/8a90830db73c49d1852256ca9d930495` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:19.945326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a90830db73c49d1852256ca9d930495)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:19.947249 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.190326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a90830db73c49d1852256ca9d930495'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:20.191050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a90830db73c49d1852256ca9d930495'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:20.211396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a90830db73c49d1852256ca9d930495'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:20.212280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a90830db73c49d1852256ca9d930495'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:20.220749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a90830db73c49d1852256ca9d930495'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:20.308556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1347/2704] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:19 2022] DELETE /identity/v3/users/8a90830db73c49d1852256ca9d930495 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:20.313893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.314429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.314429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.333744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:20.334959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:20.398073 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1358/2705] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:20.408006 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:20.412489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:20.456311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:20.472936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6bde-84bb-4b31-b051-45c266a39163 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:20.476959 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:20.477589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:20.477879 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:20.478153 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:20.488762 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:20.490271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:20.499896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1348/2706] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:21.133848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:21.135408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:21.153116 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ebca-651a-4c53-a8d6-91f27e3ed374 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:21.157411 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:21.158239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:21.158773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:21.158773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] PATH_INFO: `/v3/projects/f8ea8b9fda9c47d194a46e7372305932` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:21.160059 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ea8b9fda9c47d194a46e7372305932)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:21.170747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:21.331173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ea8b9fda9c47d194a46e7372305932', 'request_id': 'req-8ffde316-1d45-4361-8a03-cdc5936c6378'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.337506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ea8b9fda9c47d194a46e7372305932', 'request_id': 'req-8ffde316-1d45-4361-8a03-cdc5936c6378'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.445539 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ea8b9fda9c47d194a46e7372305932', 'request_id': 'req-8ffde316-1d45-4361-8a03-cdc5936c6378'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.460093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] The token cache is being invalidate because project f8ea8b9fda9c47d194a46e7372305932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:21.460543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.461128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.462669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1359/2707] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:21 2022] DELETE /identity/v3/projects/f8ea8b9fda9c47d194a46e7372305932 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:21.467706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:21.479735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9ab3-0ef5-445e-a7e3-89064d5f929b service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:21.541265 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:21.542126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:21.542334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:21.542672 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] PATH_INFO: `/v3/users/b8af2e22c549476e9bd0c4b8379b31e7` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:21.544759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8af2e22c549476e9bd0c4b8379b31e7)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:21.546809 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:21.803005 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8af2e22c549476e9bd0c4b8379b31e7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.804063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8af2e22c549476e9bd0c4b8379b31e7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.819704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8af2e22c549476e9bd0c4b8379b31e7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.820815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8af2e22c549476e9bd0c4b8379b31e7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.828189 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-43ee0f9a-f541-4e56-b42c-089606f3cb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8af2e22c549476e9bd0c4b8379b31e7'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:21.964174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1349/2708] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:21 2022] DELETE /identity/v3/users/b8af2e22c549476e9bd0c4b8379b31e7 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:22.447012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:22.448544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:22.466689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde316-1d45-4361-8a03-cdc5936c6378 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:22.472598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:22.536578 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:22.537251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:22.537564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:22.537863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] PATH_INFO: `/v3/projects/9fc31a323f104a38a9c8c7f99131fc0e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:22.539038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc31a323f104a38a9c8c7f99131fc0e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:22.541061 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:22.678286 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc31a323f104a38a9c8c7f99131fc0e', 'request_id': 'req-813c5fac-3bdc-4bc5-9544-d413b3109f57'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:22.685737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc31a323f104a38a9c8c7f99131fc0e', 'request_id': 'req-813c5fac-3bdc-4bc5-9544-d413b3109f57'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:22.771274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc31a323f104a38a9c8c7f99131fc0e', 'request_id': 'req-813c5fac-3bdc-4bc5-9544-d413b3109f57'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:22.790350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] The token cache is being invalidate because project 9fc31a323f104a38a9c8c7f99131fc0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:22.790933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:22.794908 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-813c5fac-3bdc-4bc5-9544-d413b3109f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:22.797366 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1360/2709] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:22 2022] DELETE /identity/v3/projects/9fc31a323f104a38a9c8c7f99131fc0e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:22.804151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:22.804634 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:22.804722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:22.830034 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:22.832142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:22.912569 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1350/2710] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:22.920330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:22.924089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:22.924089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:22.949825 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:22.951637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:22.960902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1361/2711] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:22.965190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:22.970872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:22.987225 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92e49-1686-4aa8-aa1d-0164b8d03948 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:22.990318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:22.990936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:22.991216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:22.991484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:22.992165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:22.992471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:22.994137 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.009177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1351/2712] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:23.014601 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:23.024655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db1022-cda7-4f91-bca7-733bd507b8dc None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:23.034327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:23.035086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.035396 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.035734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.036533 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:23.038529 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.178121 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1362/2713] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:23 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:23.184392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:23.195682 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb947960-c1a0-4b7f-b841-46a2cdfa4c63 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:23.256639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:23.257176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.257461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.257740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.258425 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:23.260401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.417352 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1352/2714] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:23 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:23.422485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:23.434316 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cef8c9-4ad4-4d94-a7e8-798fdfc7b38d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:23.439844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:23.440489 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.440774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.441060 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.441739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:23.443414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.454343 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1363/2715] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:23.458390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:23.474276 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cc34c-e667-45f6-9b93-50ddd8794f36 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:23.480398 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc2c24-6c69-43be-b506-ebf8d5ef9052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:23.481025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc2c24-6c69-43be-b506-ebf8d5ef9052 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.483280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc2c24-6c69-43be-b506-ebf8d5ef9052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.483280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc2c24-6c69-43be-b506-ebf8d5ef9052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.483280 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc2c24-6c69-43be-b506-ebf8d5ef9052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:23.484632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc2c24-6c69-43be-b506-ebf8d5ef9052 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.496264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1353/2716] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:23.500744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:23.511757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa896a-21f4-4efc-a9f7-c42d14687623 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:23.527973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:23.527973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.527973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.527973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] PATH_INFO: `/v3/projects/9cab9676943845208767155371710183/users/d0d24763959841d685b99e9c3bd558a9/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.534458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cab9676943845208767155371710183, user_id=d0d24763959841d685b99e9c3bd558a9, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:23.537227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.665388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1364/2717] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:23 2022] PUT /identity/v3/projects/9cab9676943845208767155371710183/users/d0d24763959841d685b99e9c3bd558a9/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:23.677643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.677954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.678227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.706730 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] MFA Rules not processed for user `d0d24763959841d685b99e9c3bd558a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:23.708040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:23.775561 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1354/2718] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:23.786702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:23.788518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:23.829551 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:23.848036 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9546c-2c13-4573-9b22-6543c6431996 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:23.851863 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9beae7-a01a-4c6c-99b0-c082ec49d197 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ed41f2207eb4938b26f287a025f4bcc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:23.852557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9beae7-a01a-4c6c-99b0-c082ec49d197 service glance] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:23.852839 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9beae7-a01a-4c6c-99b0-c082ec49d197 service glance] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:23.853117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9beae7-a01a-4c6c-99b0-c082ec49d197 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:23.862731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9beae7-a01a-4c6c-99b0-c082ec49d197 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:23.864215 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9beae7-a01a-4c6c-99b0-c082ec49d197 service glance] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:23.873199 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1365/2719] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:27.196018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:27.197451 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:27.262554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:27.267051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6b83d-2794-40d2-809d-5022bb5c683b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:27.271952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:27.275018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:27.275018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:27.275018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] PATH_INFO: `/v3/users/cdef060ee45e4bf58cc8e81fb7f61f63` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:27.275018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdef060ee45e4bf58cc8e81fb7f61f63)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:27.276601 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:27.496860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdef060ee45e4bf58cc8e81fb7f61f63'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:27.497501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdef060ee45e4bf58cc8e81fb7f61f63'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:27.509571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdef060ee45e4bf58cc8e81fb7f61f63'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:27.510243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdef060ee45e4bf58cc8e81fb7f61f63'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:27.517247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdef060ee45e4bf58cc8e81fb7f61f63'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:27.592402 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1355/2720] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:27 2022] DELETE /identity/v3/users/cdef060ee45e4bf58cc8e81fb7f61f63 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:27.597307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:27.597841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:27.597841 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:27.617798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:27.619062 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:27.684976 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1366/2721] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:27.695264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:27.697124 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:27.759508 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bc20b-a961-4801-994e-8337a136f656 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:27.763087 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b49b9-cba2-4f0d-8356-86faa945dd73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:27.763702 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b49b9-cba2-4f0d-8356-86faa945dd73 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:27.764305 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b49b9-cba2-4f0d-8356-86faa945dd73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:27.764579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b49b9-cba2-4f0d-8356-86faa945dd73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:27.773946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b49b9-cba2-4f0d-8356-86faa945dd73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:27.778544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b49b9-cba2-4f0d-8356-86faa945dd73 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:27.785172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1356/2722] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:35:28.327405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:28.329126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:28.346797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a85d6-3d62-4896-a8c7-88f4a7f02a99 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:28.351905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:28.352416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:28.352699 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:28.352974 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] PATH_INFO: `/v3/projects/abd71d237e054c94aba2af45655b6632` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:28.354277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd71d237e054c94aba2af45655b6632)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:28.356843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:28.486486 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd71d237e054c94aba2af45655b6632', 'request_id': 'req-d6c91d53-5f5e-4660-9a2f-008790bf42e9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:28.492260 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd71d237e054c94aba2af45655b6632', 'request_id': 'req-d6c91d53-5f5e-4660-9a2f-008790bf42e9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:28.579409 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd71d237e054c94aba2af45655b6632', 'request_id': 'req-d6c91d53-5f5e-4660-9a2f-008790bf42e9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:28.592710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] The token cache is being invalidate because project abd71d237e054c94aba2af45655b6632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:28.594403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:28.594403 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d6c91d53-5f5e-4660-9a2f-008790bf42e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:28.595447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1367/2723] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:28 2022] DELETE /identity/v3/projects/abd71d237e054c94aba2af45655b6632 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:28.600710 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:28.601216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:28.601336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:28.621485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:28.622815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:28.689295 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1357/2724] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:28.697235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:28.697803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:28.697803 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:28.718535 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:28.720684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:28.728906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1368/2725] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:28.733525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:28.744580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21e0fc-c3e2-4d43-8104-05686386ecf6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:28.749873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:28.750485 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:28.750771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:28.751190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:28.751843 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:28.752187 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:28.753841 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:28.766685 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1358/2726] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:28.771719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:28.780804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562c00d-b648-4f80-a12a-d8df179f4558 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:28.785632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:28.786239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:28.786522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:28.786797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:28.787438 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:28.789413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:28.904331 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1369/2727] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:28 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:28.908948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:28.919709 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9b765-e38f-4f4a-9b4e-f67534f2fbc7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:28.923688 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:28.977937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:28.978642 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:28.978933 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:28.979247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:28.979868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:28.982148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:29.080380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1359/2728] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:28 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:29.085959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:29.097397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e0001-8b13-4662-836f-15c6a4c70470 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:29.102196 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:29.102802 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:29.103180 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:29.103457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:29.104133 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:29.105747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:29.116667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1370/2729] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:29.121097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:29.134003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987ce70-c991-49c9-9aeb-8c7346f8d0c2 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:29.139050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2140ff-ff1a-44a2-9970-45e4514ea1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:29.139639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2140ff-ff1a-44a2-9970-45e4514ea1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:29.139981 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2140ff-ff1a-44a2-9970-45e4514ea1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:29.140258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2140ff-ff1a-44a2-9970-45e4514ea1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:29.140902 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2140ff-ff1a-44a2-9970-45e4514ea1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:29.142548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2140ff-ff1a-44a2-9970-45e4514ea1b0 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:29.155625 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1360/2730] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:29.159732 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:29.170873 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54b284-c8bd-4fe0-93ee-67f8a2e61a40 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:29.175455 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:29.176144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:29.176443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:29.176844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] PATH_INFO: `/v3/projects/ecd13ced46854940b3765ee641390212/users/6576ed5b370049f5bd091512d9803d3c/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:29.197005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd13ced46854940b3765ee641390212, user_id=6576ed5b370049f5bd091512d9803d3c, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:29.199652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:29.269044 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:29.274597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1371/2731] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:29 2022] PUT /identity/v3/projects/ecd13ced46854940b3765ee641390212/users/6576ed5b370049f5bd091512d9803d3c/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:29.279746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e04f-1c8f-493b-a18f-68131539ca99 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:29.280074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e04f-1c8f-493b-a18f-68131539ca99 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:29.282084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e04f-1c8f-493b-a18f-68131539ca99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:29.299940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-67e9e04f-1c8f-493b-a18f-68131539ca99 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:29.301172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-67e9e04f-1c8f-493b-a18f-68131539ca99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:29.369081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1361/2732] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:29.379655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:29.381135 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:29.439637 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66205b-a481-444c-815d-45de3792fb88 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:29.443204 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:29.443812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:29.444123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:29.444401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:29.454107 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:29.455627 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:29.466523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1372/2733] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:35.965751 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:35.966588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:35.966869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:35.997825 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] MFA Rules not processed for user `6576ed5b370049f5bd091512d9803d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:35.999068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:36.071691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1362/2734] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:36.076401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.089257 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1889a22-0c68-460e-9f5c-6ae1bff3e76e service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.095258 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.096038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.096209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.096482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.097123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.099208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.212450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1373/2735] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:36.214114 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.225908 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a9c99-729a-40ae-b179-9aacd23783bb None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.239369 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:36.290320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.290941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.291227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.291528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.292236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.294264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.413808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1363/2736] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:36 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:36.417941 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.428759 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af9cd4-cfcf-466c-9f14-cd5b0f006c5d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.433338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.434510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.434810 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.435081 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.435725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.437351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.450571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1374/2737] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:36.455240 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.467138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c859d0c-60e6-42be-9ed7-16639ff9dc86 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.472651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.473346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.474387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.474387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] PATH_INFO: `/v3/projects/41023f7417314c0b9e63e0d68bbb7d2f/users/80a644b6ea534e75a529f5514a9be68c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.496078 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41023f7417314c0b9e63e0d68bbb7d2f, user_id=80a644b6ea534e75a529f5514a9be68c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.499362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.589172 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1364/2738] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:36 2022] PUT /identity/v3/projects/41023f7417314c0b9e63e0d68bbb7d2f/users/80a644b6ea534e75a529f5514a9be68c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:36.593416 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.605405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415fe5-8e91-42e4-9bcd-1b64ed137ec5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.678268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462106a4-fee6-428a-afb8-de72dd05cc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.678842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462106a4-fee6-428a-afb8-de72dd05cc82 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.679272 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462106a4-fee6-428a-afb8-de72dd05cc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.679458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462106a4-fee6-428a-afb8-de72dd05cc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.680149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462106a4-fee6-428a-afb8-de72dd05cc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.682301 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462106a4-fee6-428a-afb8-de72dd05cc82 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.698528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1375/2739] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:36.702406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:36.711543 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6148d7d-7cdf-49db-88f6-124fe1a09481 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:36.717269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:36.717967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:36.718250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:36.718528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] PATH_INFO: `/v3/projects/41023f7417314c0b9e63e0d68bbb7d2f/users/80a644b6ea534e75a529f5514a9be68c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:36.722012 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41023f7417314c0b9e63e0d68bbb7d2f, user_id=80a644b6ea534e75a529f5514a9be68c, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:36.724776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:36.737392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1365/2740] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:36 2022] PUT /identity/v3/projects/41023f7417314c0b9e63e0d68bbb7d2f/users/80a644b6ea534e75a529f5514a9be68c/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:35:43.450569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:43.451157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:43.451157 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:43.483683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] MFA Rules not processed for user `80a644b6ea534e75a529f5514a9be68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:43.485018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:43.550306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:43.578181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1376/2741] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:43.598645 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:43.600756 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:43.727023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26f4db-7683-4fa6-ab86-68aac61f3888 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:43.733323 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c60e9-ed9e-4b1b-9d53-b97ca1d0c2b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:43.733948 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c60e9-ed9e-4b1b-9d53-b97ca1d0c2b8 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:43.734285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c60e9-ed9e-4b1b-9d53-b97ca1d0c2b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:43.734604 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c60e9-ed9e-4b1b-9d53-b97ca1d0c2b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:43.780886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:43.780886 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:43.782518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:43.802750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c60e9-ed9e-4b1b-9d53-b97ca1d0c2b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:43.804654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c60e9-ed9e-4b1b-9d53-b97ca1d0c2b8 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:43.818413 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1366/2742] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:43.882447 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a496a4-a12c-4e83-bb24-4c3f6be269b2 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:43.887978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:43.896565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:43.896565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:43.896565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] PATH_INFO: `/v3/users/95a453adcd8d486a952f1dd79d642ff3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:43.896565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a453adcd8d486a952f1dd79d642ff3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:43.896565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:44.204344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a453adcd8d486a952f1dd79d642ff3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:44.278830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a453adcd8d486a952f1dd79d642ff3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:44.287371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a453adcd8d486a952f1dd79d642ff3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:44.301811 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a453adcd8d486a952f1dd79d642ff3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:44.302550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a453adcd8d486a952f1dd79d642ff3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:44.312754 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1377/2743] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:43 2022] DELETE /identity/v3/users/95a453adcd8d486a952f1dd79d642ff3 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:44.314998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:44.315486 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:44.315771 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:44.337224 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:44.338546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:44.406309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1367/2744] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:44.417466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:44.418997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:44.487800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee4383-fc47-456d-b079-3cf52aa5af9e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:44.492458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:44.492458 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:44.492746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:44.492846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:44.504775 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:44.506483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:44.520104 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1378/2745] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:45.030541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:45.032776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:45.045464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb855cb-23a8-420f-99dc-140e5b6b88be None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:45.050565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:45.051236 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.051523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.051807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] PATH_INFO: `/v3/users/d0d24763959841d685b99e9c3bd558a9` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.056945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d24763959841d685b99e9c3bd558a9)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:45.059221 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:45.081929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:45.081929 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:45.103361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05319e22-cd6c-41dc-a659-5fb59a5b8c6d service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:45.116973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:45.116973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.116973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.116973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] PATH_INFO: `/v3/projects/11169e8b127044a09efa5640e8fc54f7` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.117459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] RBAC: Authorizing `identity:delete_project(project_id=11169e8b127044a09efa5640e8fc54f7)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:45.118189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:45.125220 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:45.391023 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d24763959841d685b99e9c3bd558a9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.392620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11169e8b127044a09efa5640e8fc54f7', 'request_id': 'req-00e8bc40-704a-4ea5-81da-29b1ca79336d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.392769 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d24763959841d685b99e9c3bd558a9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.396977 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11169e8b127044a09efa5640e8fc54f7', 'request_id': 'req-00e8bc40-704a-4ea5-81da-29b1ca79336d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.404341 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d24763959841d685b99e9c3bd558a9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.405143 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d24763959841d685b99e9c3bd558a9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.412316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-954f2e4b-d89a-4944-be04-3a52a05f5f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d24763959841d685b99e9c3bd558a9'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.479100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11169e8b127044a09efa5640e8fc54f7', 'request_id': 'req-00e8bc40-704a-4ea5-81da-29b1ca79336d'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.491254 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] The token cache is being invalidate because project 11169e8b127044a09efa5640e8fc54f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:45.492307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.492980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-00e8bc40-704a-4ea5-81da-29b1ca79336d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:45.494505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1379/2746] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:45 2022] DELETE /identity/v3/projects/11169e8b127044a09efa5640e8fc54f7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:45.499913 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091b284-f56a-4bdb-af3c-28079d13a447 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.500395 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091b284-f56a-4bdb-af3c-28079d13a447 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.500778 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091b284-f56a-4bdb-af3c-28079d13a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.523647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-d091b284-f56a-4bdb-af3c-28079d13a447 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:45.524916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d091b284-f56a-4bdb-af3c-28079d13a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:45.586435 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1380/2747] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:45.595110 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.595577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.596125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.633994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:45.635662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:45.641605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1368/2748] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:35:45 2022] DELETE /identity/v3/users/d0d24763959841d685b99e9c3bd558a9 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:45.646381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.647052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.647332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.649791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1381/2749] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:45.657017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:45.667534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:45.668967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695862c3-0093-4286-9018-04a5ec81fae3 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:45.669717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:45.724964 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:45.744204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1369/2750] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:45.754866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:45.756994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:45.757153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:45.757501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.758013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.758477 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.759142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:45.759608 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:45.761492 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:45.780796 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1382/2751] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:35:45.786098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:45.797336 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db524-8582-4d6c-9506-c8f36493d6df admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:45.804088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:45.804716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.805244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.805674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.806491 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:45.809365 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:45.835408 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf70b44-6c74-4b3f-9c62-cd404337506b None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:45.844480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:45.845251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:45.845534 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:45.845806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:45.859818 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:45.861443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:45.872745 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1370/2752] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:45.953887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1383/2753] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:45 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:45.957901 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:45.977608 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6181c7e-a3fd-4f5b-b931-11b8e4200187 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:46.045496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:46.046188 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.046396 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.046674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.047363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:46.049409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:46.146476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1371/2754] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:45 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:46.147124 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:46.161208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db3629-0504-4c92-adf8-221cefe5ae5f admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:46.167521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:46.169103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.169543 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.170935 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.171952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:46.174512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:46.199113 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1384/2755] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:46.200640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:46.224808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2345f9-5752-4229-a80a-e4eccdc6b733 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:46.247128 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e9a85-d8ab-4bbe-a03d-f82feac18e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:46.248105 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e9a85-d8ab-4bbe-a03d-f82feac18e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.249061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e9a85-d8ab-4bbe-a03d-f82feac18e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.249392 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e9a85-d8ab-4bbe-a03d-f82feac18e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.250129 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e9a85-d8ab-4bbe-a03d-f82feac18e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:46.251844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e9a85-d8ab-4bbe-a03d-f82feac18e6c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:46.264674 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1372/2756] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:46.268799 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:46.289521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ffe64-2b8f-4efd-bb1f-a38293db9037 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:46.295546 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:46.296349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.296701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.297050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] PATH_INFO: `/v3/projects/95e230e152ef4693a1e796f87c8e78e1/users/b41fffef80a44eeaabd461ade780058a/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.314357 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e230e152ef4693a1e796f87c8e78e1, user_id=b41fffef80a44eeaabd461ade780058a, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:46.317437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:46.481077 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1385/2757] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:46 2022] PUT /identity/v3/projects/95e230e152ef4693a1e796f87c8e78e1/users/b41fffef80a44eeaabd461ade780058a/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:46.491433 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:46.492325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.492623 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.492903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.506264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:46.509512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:46.542767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] MFA Rules not processed for user `b41fffef80a44eeaabd461ade780058a`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:46.542767 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:46.639427 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1373/2758] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:46.650991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:46.654119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:46.677985 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c262ac5-7f23-4111-a5f8-777f68783d7c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:46.683700 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:46.684860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.685421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.685859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] PATH_INFO: `/v3/projects/9cab9676943845208767155371710183` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.687377 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cab9676943845208767155371710183)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:46.689966 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:46.801059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded9259-098b-4f8e-a66f-06449ddb5833 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:46.809907 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e6024-7802-49ac-bcf5-c7b8d37188f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:46.811099 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6024-7802-49ac-bcf5-c7b8d37188f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:46.811723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6024-7802-49ac-bcf5-c7b8d37188f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:46.812348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e6024-7802-49ac-bcf5-c7b8d37188f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:46.831448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6024-7802-49ac-bcf5-c7b8d37188f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:46.834606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e6024-7802-49ac-bcf5-c7b8d37188f8 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:46.854397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1374/2759] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:46.888519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cab9676943845208767155371710183', 'request_id': 'req-0a2a3239-b0be-4578-a612-40bb79ddedbc'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:46.901887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cab9676943845208767155371710183', 'request_id': 'req-0a2a3239-b0be-4578-a612-40bb79ddedbc'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:47.096400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cab9676943845208767155371710183', 'request_id': 'req-0a2a3239-b0be-4578-a612-40bb79ddedbc'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:47.116450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] The token cache is being invalidate because project 9cab9676943845208767155371710183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:35:47.117035 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:47.117919 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-0a2a3239-b0be-4578-a612-40bb79ddedbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:35:47.119662 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1386/2760] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:35:46 2022] DELETE /identity/v3/projects/9cab9676943845208767155371710183 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:47.126754 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.127953 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.128525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.157941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:47.159222 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:47.248884 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1375/2761] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:47.257050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.257327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.257610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.305086 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:47.306782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:47.319024 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1387/2762] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:47.323613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:47.339591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c7a8f-04f7-4f4d-bebf-0aeea43321b6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:47.349088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:47.350218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.350740 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.351257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.360052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:35:47.363367 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:47.364589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:47.392056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:47.403958 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1376/2763] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:35:47.406232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:47.440564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b4458-33f0-486f-8833-47dd725609f7 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:47.460753 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ca602-07cd-4fe8-9082-490b87935d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:47.461412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ca602-07cd-4fe8-9082-490b87935d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.461741 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ca602-07cd-4fe8-9082-490b87935d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.462014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ca602-07cd-4fe8-9082-490b87935d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.462677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ca602-07cd-4fe8-9082-490b87935d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:47.465236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ca602-07cd-4fe8-9082-490b87935d5c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:47.598655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1388/2764] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:47 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:47.600480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:47.614052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eea91-a8d1-47a6-b668-65df06e0b210 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:47.689360 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:47.690002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.690286 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.690518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.690628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.691023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.691211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:47.691519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.693325 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:47.720483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] MFA Rules not processed for user `6576ed5b370049f5bd091512d9803d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:47.722175 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:47.797870 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1389/2765] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:47.897562 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1377/2766] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:47 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:47.902632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:47.920800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece73c7-7366-4241-80db-71209fc9cde9 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:47.927137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:47.927867 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.928274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.928740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.929582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:47.932295 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:47.944707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1390/2767] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:47.949266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:47.961239 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2f92-7670-4d10-87f5-e02e2ee94c30 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:47.969313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4dda8-9d28-4f71-8372-53e04fe076c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:47.970301 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4dda8-9d28-4f71-8372-53e04fe076c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:47.970768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4dda8-9d28-4f71-8372-53e04fe076c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:47.971227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4dda8-9d28-4f71-8372-53e04fe076c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:47.972438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4dda8-9d28-4f71-8372-53e04fe076c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:47.975719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4dda8-9d28-4f71-8372-53e04fe076c5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:48.004479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1378/2768] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:48.005423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:48.032488 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ed33b-e1e3-4fe5-9c10-25c5b3918044 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:48.044846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:48.046095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:48.046846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:48.047868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] PATH_INFO: `/v3/projects/8cb8fd6ded3d4a56b06ed52791c4ee47/users/ba8ee62ef37a4e98968577413339280d/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:48.079430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb8fd6ded3d4a56b06ed52791c4ee47, user_id=ba8ee62ef37a4e98968577413339280d, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:48.084805 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:48.252511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1391/2769] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:48 2022] PUT /identity/v3/projects/8cb8fd6ded3d4a56b06ed52791c4ee47/users/ba8ee62ef37a4e98968577413339280d/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:48.265281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:48.266363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:48.266363 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:48.308458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] MFA Rules not processed for user `ba8ee62ef37a4e98968577413339280d`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:48.310045 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:48.522706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1379/2770] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:48.528525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:48.624136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4d610-5325-428e-b6c6-8162821fb5a9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:48.630040 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:48.630682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:48.630961 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:48.631241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:48.631967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:48.633957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:48.694210 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:48.815522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1392/2771] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:48 2022] POST /identity/v3/projects => generated 385 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:48.815732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:48.829362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d352217-f06a-4982-9f02-01356fc934c4 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:48.898340 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:48.899264 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:48.899621 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:48.900009 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:48.907828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:48.907828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:48.935357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:49.065125 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1380/2772] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:48 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:35:49.069631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:49.083079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad71f4d-be83-48a3-be19-df3bea9c9685 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:49.088464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:49.089902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.090242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.090523 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.091484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:49.093206 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:49.107399 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1393/2773] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:49.115257 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:49.129191 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca79f76-195a-42ed-acf6-8a0b881e20ce admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:49.138868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd45d85-a91f-45b4-be52-273d1abf5a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:49.139501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd45d85-a91f-45b4-be52-273d1abf5a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.139780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd45d85-a91f-45b4-be52-273d1abf5a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.140097 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd45d85-a91f-45b4-be52-273d1abf5a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.140748 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd45d85-a91f-45b4-be52-273d1abf5a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:49.142362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd45d85-a91f-45b4-be52-273d1abf5a21 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:49.154701 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1381/2774] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:49.159555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:49.170468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bc085-cb52-4fa4-95bd-858db5f75658 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:49.176250 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d0c62-7fdc-49e1-bfc3-f3b80fd258a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:49.177037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0c62-7fdc-49e1-bfc3-f3b80fd258a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.177386 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0c62-7fdc-49e1-bfc3-f3b80fd258a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.177736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d0c62-7fdc-49e1-bfc3-f3b80fd258a0 admin admin] PATH_INFO: `/v3/projects/af01aee140924f509121808cb54e3220/users/f575423dd4eb4b9fb286e7a561605854/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.199758 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0c62-7fdc-49e1-bfc3-f3b80fd258a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af01aee140924f509121808cb54e3220, user_id=f575423dd4eb4b9fb286e7a561605854, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:49.202823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0d0c62-7fdc-49e1-bfc3-f3b80fd258a0 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:49.231704 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.232040 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.232316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.253030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] MFA Rules not processed for user `b41fffef80a44eeaabd461ade780058a`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:49.257120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:49.293695 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1394/2775] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:49 2022] PUT /identity/v3/projects/af01aee140924f509121808cb54e3220/users/f575423dd4eb4b9fb286e7a561605854/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:49.315954 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1382/2776] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:49.316588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.316588 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.316896 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.351738 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] MFA Rules not processed for user `f575423dd4eb4b9fb286e7a561605854`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:49.353136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:49.443564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1395/2777] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:35:49.449285 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:49.466197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b24650-29ba-4a11-8eed-e93291e3e8e3 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:49.562832 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:49.563965 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.564480 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.565036 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.566288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:49.570875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:49.704474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1383/2778] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:35:49 2022] POST /identity/v3/projects => generated 383 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:49.708969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:49.728674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b381c-8f79-42ce-bbd3-78c5ed4471a8 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:49.813696 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:49.814362 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:49.814657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:49.814940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:49.815613 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:49.817829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:49.852759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:49.977000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1396/2779] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:35:49 2022] POST /identity/v3/users => generated 298 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:35:49.981939 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:49.987612 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:49.994226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504c31f-d1fb-47e3-ae44-ca01c1a2a87f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:50.001313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:50.001313 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:50.001723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:50.002069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:50.002861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:50.004692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:50.016992 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1384/2780] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:35:50.024118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:50.036801 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f230a-6fd8-48ce-9090-fe9f906e0b19 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:50.042565 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:50.044547 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:50.049356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:50.049356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] PATH_INFO: `/v3/projects/d77d0bf0ac0c427aac552c1e6361687e/users/dde1b6050e714a918da34f51a5abc02a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:50.078933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77d0bf0ac0c427aac552c1e6361687e, user_id=dde1b6050e714a918da34f51a5abc02a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:50.083689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:50.178981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1397/2781] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:50 2022] PUT /identity/v3/projects/d77d0bf0ac0c427aac552c1e6361687e/users/dde1b6050e714a918da34f51a5abc02a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:50.183115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:50.195474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bbc46-5176-4a66-bd7b-97e7c9314004 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:50.255615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02f8c-d488-45db-b6fd-484b68b2e8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:50.256242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02f8c-d488-45db-b6fd-484b68b2e8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:50.256523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02f8c-d488-45db-b6fd-484b68b2e8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:50.256796 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02f8c-d488-45db-b6fd-484b68b2e8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:50.257589 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a02f8c-d488-45db-b6fd-484b68b2e8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:50.259192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a02f8c-d488-45db-b6fd-484b68b2e8f9 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:50.272311 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1385/2782] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:35:50.282942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:50.294672 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7d0c2-aabb-4e6b-8241-dd56d9c0b888 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:50.300420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:50.301465 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:50.301814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:50.302209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] PATH_INFO: `/v3/projects/d77d0bf0ac0c427aac552c1e6361687e/users/dde1b6050e714a918da34f51a5abc02a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:50.306289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d77d0bf0ac0c427aac552c1e6361687e, user_id=dde1b6050e714a918da34f51a5abc02a, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:50.309176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:50.322635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1398/2783] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:35:50 2022] PUT /identity/v3/projects/d77d0bf0ac0c427aac552c1e6361687e/users/dde1b6050e714a918da34f51a5abc02a/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:35:50.334856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:50.335433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:50.335433 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:50.374797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] MFA Rules not processed for user `dde1b6050e714a918da34f51a5abc02a`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:35:50.374797 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:50.458532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1386/2784] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:35:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:35:50.471857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:50.474162 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:35:50.542393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222e92a-1c61-44f2-ab94-58da425da71d admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:50.546299 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:50.546964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:50.547300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:50.548449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:50.611105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:50.612594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:50.623249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1399/2785] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:50.704926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:52.955227 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:52.966600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e7b19-0b7a-495e-ab74-c55a495e5b2e None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:52.970735 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:52.971355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:52.971736 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:52.972061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:52.992158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:35:53.038489 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:53.040173 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:53.052008 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1387/2786] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:35:55.069569 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:35:55.081773 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb934b1-9442-4dcf-a8f0-2a8a4a0c5a12 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:35:55.087675 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83319c19-16ee-4fd9-b42d-a0c397c42f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:35:55.088668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83319c19-16ee-4fd9-b42d-a0c397c42f12 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:35:55.089023 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83319c19-16ee-4fd9-b42d-a0c397c42f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:35:55.090277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83319c19-16ee-4fd9-b42d-a0c397c42f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:35:55.105325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83319c19-16ee-4fd9-b42d-a0c397c42f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:35:55.106931 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83319c19-16ee-4fd9-b42d-a0c397c42f12 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:35:55.118046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1400/2787] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:35:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:36:00.318779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:00.394599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ce5a7-7302-4096-b8bf-a33f5a0dd236 service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:00.399197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:00.399940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.400345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.400639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] PATH_INFO: `/v3/users/ba8ee62ef37a4e98968577413339280d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:00.402327 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8ee62ef37a4e98968577413339280d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:00.404332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:00.697659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8ee62ef37a4e98968577413339280d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.699320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8ee62ef37a4e98968577413339280d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.724503 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8ee62ef37a4e98968577413339280d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.725823 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8ee62ef37a4e98968577413339280d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.734316 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8ee62ef37a4e98968577413339280d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:00.815074 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1388/2788] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:00 2022] DELETE /identity/v3/users/ba8ee62ef37a4e98968577413339280d => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:00.822464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:00.822464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:00.831249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:00.858468 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:00.860887 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:00.952068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1401/2789] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:00.962255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:00.963696 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.027972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:01.058693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7ffb-8011-4f52-9c86-d09617a0f2cc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.066162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.067349 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.069056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.069056 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.086564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.089405 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.109934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1389/2790] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:01.690126 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:01.691853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:01.708249 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a300b9-30ca-4b20-93bb-b926d57d2d60 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:01.715012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:01.716138 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:01.716893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:01.717370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] PATH_INFO: `/v3/projects/8cb8fd6ded3d4a56b06ed52791c4ee47` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:01.719604 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb8fd6ded3d4a56b06ed52791c4ee47)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:01.721615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:01.874843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb8fd6ded3d4a56b06ed52791c4ee47', 'request_id': 'req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.881901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb8fd6ded3d4a56b06ed52791c4ee47', 'request_id': 'req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.967746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb8fd6ded3d4a56b06ed52791c4ee47', 'request_id': 'req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.981583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] The token cache is being invalidate because project 8cb8fd6ded3d4a56b06ed52791c4ee47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:01.981880 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.982781 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:01.985582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1402/2791] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:01 2022] DELETE /identity/v3/projects/8cb8fd6ded3d4a56b06ed52791c4ee47 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:01.991800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:02.005763 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1409ce-2145-45fd-a55e-d52e2506a89f service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:02.088473 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:02.089731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:02.090106 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:02.090292 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] PATH_INFO: `/v3/users/f575423dd4eb4b9fb286e7a561605854` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:02.092882 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f575423dd4eb4b9fb286e7a561605854)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:02.093669 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:02.349920 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f575423dd4eb4b9fb286e7a561605854'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.350838 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f575423dd4eb4b9fb286e7a561605854'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.370897 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f575423dd4eb4b9fb286e7a561605854'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.372441 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f575423dd4eb4b9fb286e7a561605854'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.385381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f575423dd4eb4b9fb286e7a561605854'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:02.523827 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1390/2792] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:01 2022] DELETE /identity/v3/users/f575423dd4eb4b9fb286e7a561605854 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:03.102441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.105420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:03.139454 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6db89b-3a4f-4327-9672-ed2fecbd1b3c admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.201986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:03.211623 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.212427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.212706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.212984 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] PATH_INFO: `/v3/projects/af01aee140924f509121808cb54e3220` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.214739 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af01aee140924f509121808cb54e3220)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.216771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.463303 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af01aee140924f509121808cb54e3220', 'request_id': 'req-142a95f6-89f5-43e5-9003-ae1dea4e98c9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:03.472899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af01aee140924f509121808cb54e3220', 'request_id': 'req-142a95f6-89f5-43e5-9003-ae1dea4e98c9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:03.585500 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af01aee140924f509121808cb54e3220', 'request_id': 'req-142a95f6-89f5-43e5-9003-ae1dea4e98c9'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:03.608757 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] The token cache is being invalidate because project af01aee140924f509121808cb54e3220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:03.609211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:03.610335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:03.612973 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1403/2793] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:03 2022] DELETE /identity/v3/projects/af01aee140924f509121808cb54e3220 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:03.619035 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:03.622330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:03.645275 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758914c-02b3-4ea3-8e67-076fd81d9436 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:03.750103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:03.751291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:03.751757 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:03.752253 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] PATH_INFO: `/v3/users/dde1b6050e714a918da34f51a5abc02a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:03.753941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde1b6050e714a918da34f51a5abc02a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:03.756047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:03.758259 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:04.096871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde1b6050e714a918da34f51a5abc02a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.097656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde1b6050e714a918da34f51a5abc02a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.113554 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde1b6050e714a918da34f51a5abc02a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.114432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde1b6050e714a918da34f51a5abc02a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.122443 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3ee29608-5454-4139-b93e-ae380b4f45d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde1b6050e714a918da34f51a5abc02a'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.230738 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1391/2794] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:03 2022] DELETE /identity/v3/users/dde1b6050e714a918da34f51a5abc02a => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:04.715597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:04.717514 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:04.731986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a95f6-89f5-43e5-9003-ae1dea4e98c9 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:04.797388 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:04.798053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:04.798400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:04.798638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] PATH_INFO: `/v3/projects/d77d0bf0ac0c427aac552c1e6361687e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:04.800114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d77d0bf0ac0c427aac552c1e6361687e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:04.802128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:04.977065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd77d0bf0ac0c427aac552c1e6361687e', 'request_id': 'req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:04.983067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd77d0bf0ac0c427aac552c1e6361687e', 'request_id': 'req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.096312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd77d0bf0ac0c427aac552c1e6361687e', 'request_id': 'req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.113088 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] The token cache is being invalidate because project d77d0bf0ac0c427aac552c1e6361687e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:05.113304 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.114043 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d4c12b01-b13f-4960-a3cf-71d24cfc94c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:05.115613 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1404/2795] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:04 2022] DELETE /identity/v3/projects/d77d0bf0ac0c427aac552c1e6361687e => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:05.122496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.122496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.122496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.145380 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:05.146746 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:05.225605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1392/2796] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:05.293101 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.293525 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.293743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.319615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:05.321094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:05.331831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1405/2797] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:05.336145 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.347330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9191257-90af-4483-b43c-7e38a349c11a None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.352195 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.353166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.353648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.354132 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] PATH_INFO: `/v3/domains` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.355162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81875) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:05.355695 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.358471 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.380476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1393/2798] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:36:05.384484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.394789 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f293272-ba8f-413f-a583-c6cffd63dde1 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.400166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.400812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.401125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.402003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.402668 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.404694 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.409829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:05.555986 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1406/2799] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:05 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:36:05.558196 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.569482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd0eb-121b-4965-8081-512c80c7c134 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.650047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.650047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.650047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.650047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.650536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.650536 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.660779 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:05.780768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1394/2800] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:05 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:05.782344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.793958 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4a05c-d53f-4bc4-bfb3-ed09ac7a78f2 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.799172 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.800413 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.800540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.800780 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.801499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.803102 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.819930 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1407/2801] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:05.825348 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.836938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5e0ab-c8b2-41dc-abb9-9d0d08f480d5 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.842752 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24def3d-4166-4de2-ad2e-9014381adbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.843174 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24def3d-4166-4de2-ad2e-9014381adbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.843490 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24def3d-4166-4de2-ad2e-9014381adbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.843714 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24def3d-4166-4de2-ad2e-9014381adbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.844670 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24def3d-4166-4de2-ad2e-9014381adbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.846495 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24def3d-4166-4de2-ad2e-9014381adbb5 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:05.860697 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1395/2802] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:05.866063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:05.876998 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccc79c-7ea6-40f8-b251-cb41e854a90a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:05.881740 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:05.882398 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:05.882677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:05.882959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] PATH_INFO: `/v3/projects/44524486fb2641c7904cc6907bead070/users/a8d0f7879e4b4af9b1982b1c371d1dc0/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:05.899387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] RBAC: Authorizing `identity:create_grant(project_id=44524486fb2641c7904cc6907bead070, user_id=a8d0f7879e4b4af9b1982b1c371d1dc0, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:05.902195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:06.014978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1408/2803] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:05 2022] PUT /identity/v3/projects/44524486fb2641c7904cc6907bead070/users/a8d0f7879e4b4af9b1982b1c371d1dc0/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:06.025339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.025626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.025912 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.061936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] MFA Rules not processed for user `a8d0f7879e4b4af9b1982b1c371d1dc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:06.063650 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:06.167918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1396/2804] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:06.172122 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:06.253838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d5d73-8fc3-49ac-b8aa-638664cd391a admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:06.256229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:06.256864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.257142 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.257427 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.258183 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:06.260200 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:06.413079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1409/2805] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:06 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:06.414845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:06.427371 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052e310-09bb-4c11-846c-6c069a33de88 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:06.497107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:06.497743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.498033 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.498300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.498934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:06.501081 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:06.636069 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1397/2806] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:06 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:06.640140 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:06.653306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70ba26-9bcf-4b26-897e-0554c86ca340 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:06.658687 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:06.659402 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.659774 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.660227 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.660970 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:06.662691 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:06.681019 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1410/2807] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:06.682207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:06.695115 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbb115-8d27-4bfd-bca8-6f2d80f0aece admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:06.700988 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871222d-d92c-45b0-9c78-8f8597711c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:06.701727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871222d-d92c-45b0-9c78-8f8597711c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.701999 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871222d-d92c-45b0-9c78-8f8597711c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.702168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871222d-d92c-45b0-9c78-8f8597711c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.702824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871222d-d92c-45b0-9c78-8f8597711c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:06.704535 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871222d-d92c-45b0-9c78-8f8597711c34 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:06.716725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1398/2808] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:06.721483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:06.731968 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19741e-f9db-457f-9afa-60511d810247 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:06.736599 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:06.737271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.737550 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.737823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] PATH_INFO: `/v3/projects/4a1bcdd3b4224cfc8c1f62f047367701/users/d0faa3e3e924487891ee9cb5e7fff35f/roles/87804c58f0314370a20ec138295f6863` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.754173 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1bcdd3b4224cfc8c1f62f047367701, user_id=d0faa3e3e924487891ee9cb5e7fff35f, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:06.757001 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:06.853901 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1411/2809] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:06 2022] PUT /identity/v3/projects/4a1bcdd3b4224cfc8c1f62f047367701/users/d0faa3e3e924487891ee9cb5e7fff35f/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:06.866066 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:06.866385 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:06.869307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:06.902361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] MFA Rules not processed for user `d0faa3e3e924487891ee9cb5e7fff35f`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:06.903749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:06.951119 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:06.977026 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1399/2810] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:06.982264 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.058031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e042a-cc06-4f06-bf2d-1ff5344cea40 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:07.063997 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:07.064660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.065031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.065225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.065951 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:07.067952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:07.073783 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:07.195410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1412/2811] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:06 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 21:36:07.200544 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.212590 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55b292-4f68-4266-a112-6dd48621d105 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:07.287639 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:07.288318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.288599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.288877 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.289584 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:07.291574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:07.417799 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1400/2812] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:07 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:07.422335 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.434925 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736033-cc01-415a-b150-4ffa7320f9cd admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:07.446275 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:07.447065 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.447384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.447666 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.448809 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:07.450370 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:07.466042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1413/2813] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:07.473025 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.485391 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d6ce6-3b49-45f6-a769-4062c31efd89 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:07.494136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:07.496633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.496880 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.497335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] PATH_INFO: `/v3/projects/9011c66879df4e0b8499aa92f31be283/users/a3a48b445d3b4bf8a56e05569a910f91/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.526641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9011c66879df4e0b8499aa92f31be283, user_id=a3a48b445d3b4bf8a56e05569a910f91, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:07.531692 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:07.631155 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1401/2814] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:07 2022] PUT /identity/v3/projects/9011c66879df4e0b8499aa92f31be283/users/a3a48b445d3b4bf8a56e05569a910f91/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:07.647208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.661899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbead1d-f992-4efa-be08-ce9165e7d195 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:07.732453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a770805-cb5c-48ea-b296-0beba32117aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:07.732952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770805-cb5c-48ea-b296-0beba32117aa admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.733235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770805-cb5c-48ea-b296-0beba32117aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.733505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770805-cb5c-48ea-b296-0beba32117aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.734150 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770805-cb5c-48ea-b296-0beba32117aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:07.735812 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770805-cb5c-48ea-b296-0beba32117aa admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:07.747530 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1414/2815] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:07.752326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.768765 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d161e-0766-499d-9bd6-c1835133940c admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:07.774518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:07.775202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.775579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.775857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] PATH_INFO: `/v3/projects/9011c66879df4e0b8499aa92f31be283/users/a3a48b445d3b4bf8a56e05569a910f91/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.778570 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9011c66879df4e0b8499aa92f31be283, user_id=a3a48b445d3b4bf8a56e05569a910f91, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:07.782102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:07.794921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1402/2816] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:07 2022] PUT /identity/v3/projects/9011c66879df4e0b8499aa92f31be283/users/a3a48b445d3b4bf8a56e05569a910f91/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:07.814010 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:07.820271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:07.820271 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:07.858361 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] MFA Rules not processed for user `a3a48b445d3b4bf8a56e05569a910f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:07.860412 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:07.954959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1415/2817] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:07.968456 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:07.970806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:08.036989 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de55d8d-a1ee-48c5-8686-8c2980de55ae admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:08.041317 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ed49-5a7d-41e4-b930-54b4ae02ba70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:08.042522 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ed49-5a7d-41e4-b930-54b4ae02ba70 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:08.042829 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ed49-5a7d-41e4-b930-54b4ae02ba70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:08.043102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ed49-5a7d-41e4-b930-54b4ae02ba70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:08.103666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:08.114690 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ed49-5a7d-41e4-b930-54b4ae02ba70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:08.116210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ed49-5a7d-41e4-b930-54b4ae02ba70 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:08.134537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1403/2818] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:10.313136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:10.314538 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:10.328511 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72ff9b-993f-4bd1-a3d4-8b4c09af5fad None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:10.332145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:10.332899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:10.333195 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:10.333448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:10.391844 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:10.395537 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:10.397027 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:10.406290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1416/2819] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:36:10.645122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b71ff-4a3a-4873-82f2-5f5934779ffe None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:10.645606 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b71ff-4a3a-4873-82f2-5f5934779ffe None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:10.645723 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b71ff-4a3a-4873-82f2-5f5934779ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:10.676110 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-713b71ff-4a3a-4873-82f2-5f5934779ffe None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:10.677326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-713b71ff-4a3a-4873-82f2-5f5934779ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:10.741039 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1404/2820] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:36:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:10.936688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:10.936688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:11.025186 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d26254-8752-428c-b302-d5f6a7a7dc15 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:11.031877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:11.032531 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:11.032818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:11.033100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:11.042431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:11.044093 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:11.052907 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1417/2821] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:36:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:36:12.027550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:12.032728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:12.032728 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:12.048703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:12.049967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:12.060864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1405/2822] 10.222.0.65 () {58 vars in 967 bytes} [Mon Aug 29 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:12.078770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:12.091918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90377e-d549-4d8b-bff3-53e2b6036643 service swift] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:12.099063 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb6ba6-bd61-4864-b094-c6bf700eb555 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:12.099715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb6ba6-bd61-4864-b094-c6bf700eb555 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:12.100038 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb6ba6-bd61-4864-b094-c6bf700eb555 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:12.100314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb6ba6-bd61-4864-b094-c6bf700eb555 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:12.111185 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb6ba6-bd61-4864-b094-c6bf700eb555 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:12.112955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb6ba6-bd61-4864-b094-c6bf700eb555 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:12.124725 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1418/2823] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 21:36:12.271691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:12.282640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc439c-d87d-4e47-8106-e4fbccd66214 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:12.286381 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:12.287151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:12.287440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:12.287713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:12.300487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:12.303255 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:12.318122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1406/2824] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:12.888414 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9ad2-4200-40cb-b84a-467da2dd656c None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:12.888701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9ad2-4200-40cb-b84a-467da2dd656c None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:12.888978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd9ad2-4200-40cb-b84a-467da2dd656c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:12.909378 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-42dd9ad2-4200-40cb-b84a-467da2dd656c None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:12.910726 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-42dd9ad2-4200-40cb-b84a-467da2dd656c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:12.918797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1419/2825] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:12.933002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:12.946684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2352b-6020-42f4-b7d6-3c42bb0d9a0c service cinder] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:12.952545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:12.953211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:12.953487 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:12.953759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:12.964500 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:12.966142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:12.976226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1407/2826] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:36:13.837632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:13.838136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:13.838276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:13.860587 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:13.861817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:13.872174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1420/2827] 10.222.0.65 () {58 vars in 967 bytes} [Mon Aug 29 21:36:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:13.884268 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:13.895937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256efec1-c4ce-4622-9586-a32ba06bbc95 service swift] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:13.901250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401852b-59b0-48e0-bf17-551338b27bec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:13.906936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401852b-59b0-48e0-bf17-551338b27bec service swift] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:13.906936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401852b-59b0-48e0-bf17-551338b27bec service swift] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:13.906936 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401852b-59b0-48e0-bf17-551338b27bec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:13.912856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401852b-59b0-48e0-bf17-551338b27bec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:13.914573 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401852b-59b0-48e0-bf17-551338b27bec service swift] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:13.923271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1408/2828] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:36:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:36:17.454892 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:17.456524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:17.535092 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a0073-0fdc-4e38-9dbd-ea52b2e54870 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:17.540312 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:17.540979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:17.541420 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:17.541932 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] PATH_INFO: `/v3/users/b41fffef80a44eeaabd461ade780058a` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:17.544159 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41fffef80a44eeaabd461ade780058a)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:17.547405 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:17.758638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41fffef80a44eeaabd461ade780058a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:17.858517 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41fffef80a44eeaabd461ade780058a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:17.864384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:17.868369 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41fffef80a44eeaabd461ade780058a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:17.887095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41fffef80a44eeaabd461ade780058a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:17.887095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41fffef80a44eeaabd461ade780058a'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:17.892549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1421/2829] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:17 2022] DELETE /identity/v3/users/b41fffef80a44eeaabd461ade780058a => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:17.897874 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:17.898700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:17.899070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:17.923436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:17.925609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.014499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1409/2830] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:18.029174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:18.032676 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.093029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:18.096516 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.114875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f972d-4851-44dc-9927-bdebe5f1df35 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:18.123540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:18.124732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:18.125136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:18.125548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] PATH_INFO: `/v3/users/a8d0f7879e4b4af9b1982b1c371d1dc0` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:18.127637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d0f7879e4b4af9b1982b1c371d1dc0)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:18.131539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:18.133325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00af3ce-d52d-4773-ab72-6b258c6d96d4 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:18.139820 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89c089-7358-47d1-a0ab-9e45b1bf8de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:18.140598 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c089-7358-47d1-a0ab-9e45b1bf8de8 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:18.140959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c089-7358-47d1-a0ab-9e45b1bf8de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:18.141313 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89c089-7358-47d1-a0ab-9e45b1bf8de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:18.153128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89c089-7358-47d1-a0ab-9e45b1bf8de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:18.154730 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89c089-7358-47d1-a0ab-9e45b1bf8de8 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:18.170955 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1422/2831] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:18.321450 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:18.332951 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d0f7879e4b4af9b1982b1c371d1dc0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.333844 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d0f7879e4b4af9b1982b1c371d1dc0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.349856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d0f7879e4b4af9b1982b1c371d1dc0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.350651 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d0f7879e4b4af9b1982b1c371d1dc0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.359618 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d0f7879e4b4af9b1982b1c371d1dc0'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.427705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1410/2832] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:18 2022] DELETE /identity/v3/users/a8d0f7879e4b4af9b1982b1c371d1dc0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:18.432735 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:18.433025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:18.433309 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:18.456100 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:18.457490 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.518859 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1423/2833] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:18.528945 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:18.529349 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:18.530412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.531128 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.545678 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a5c42-c9af-4657-a9c8-edcf25634343 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:18.550616 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:18.551648 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:18.552086 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:18.552581 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] PATH_INFO: `/v3/projects/95e230e152ef4693a1e796f87c8e78e1` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:18.554426 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e230e152ef4693a1e796f87c8e78e1)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:18.556638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:18.637638 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3f0cd-ff4d-4428-be37-0a4f6125ca91 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:18.642158 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183dc72-9b0e-496b-ac03-a28ed7ff424d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:18.642830 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183dc72-9b0e-496b-ac03-a28ed7ff424d service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:18.643114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183dc72-9b0e-496b-ac03-a28ed7ff424d service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:18.643425 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183dc72-9b0e-496b-ac03-a28ed7ff424d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:18.655571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183dc72-9b0e-496b-ac03-a28ed7ff424d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:18.657046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183dc72-9b0e-496b-ac03-a28ed7ff424d service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:18.668701 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1424/2834] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:18.721198 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e230e152ef4693a1e796f87c8e78e1', 'request_id': 'req-92d452d6-1e4a-4d40-8874-350cdd7b9414'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.729030 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e230e152ef4693a1e796f87c8e78e1', 'request_id': 'req-92d452d6-1e4a-4d40-8874-350cdd7b9414'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.808007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e230e152ef4693a1e796f87c8e78e1', 'request_id': 'req-92d452d6-1e4a-4d40-8874-350cdd7b9414'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.823242 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] The token cache is being invalidate because project 95e230e152ef4693a1e796f87c8e78e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:18.823609 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.824549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-92d452d6-1e4a-4d40-8874-350cdd7b9414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:18.826254 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1411/2835] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:18 2022] DELETE /identity/v3/projects/95e230e152ef4693a1e796f87c8e78e1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:18.831822 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:18.832232 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:18.832521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:18.855558 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:18.856967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:18.877782 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:18.934918 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1425/2836] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:19.005289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.006247 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.006532 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.036424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:19.037694 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:19.048169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1412/2837] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:19.053502 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.070380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6372fdd-c1db-4bb4-856f-1972bd3986df None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.076831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.077621 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.078085 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.078570 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.079397 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:19.080218 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.084711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.107238 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1426/2838] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:36:19.112689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.138564 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b4f1e-bac0-4bee-a750-f9374c9591b6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.145475 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.146326 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.146805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.147243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.148210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.153673 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.250995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.253685 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:19.281877 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723b1c6-d1ed-4d32-83ad-0fb6124c36c7 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.292942 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1413/2839] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:19 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:19.297162 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.308816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.309734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.310031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.310540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] PATH_INFO: `/v3/projects/44524486fb2641c7904cc6907bead070` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.314344 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44524486fb2641c7904cc6907bead070)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.321910 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.393058 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8ce03-4675-4628-817b-237d3025916d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.398842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.399526 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.399807 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.400108 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] PATH_INFO: `/v3/users` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.400871 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.402858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.421019 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=82,keystone:DELETE=12,keystone:INSERT=6 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:19.478123 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44524486fb2641c7904cc6907bead070', 'request_id': 'req-bbfbd228-9e8e-44ea-a190-ba7987ba5075'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:19.484813 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44524486fb2641c7904cc6907bead070', 'request_id': 'req-bbfbd228-9e8e-44ea-a190-ba7987ba5075'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:19.529165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1414/2840] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:19 2022] POST /identity/v3/users => generated 291 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:19.533911 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.545703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2b14c-57f1-4d87-8d3f-a520dc1cf447 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.619394 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.620515 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.620888 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.621009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44524486fb2641c7904cc6907bead070', 'request_id': 'req-bbfbd228-9e8e-44ea-a190-ba7987ba5075'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:19.621365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.622429 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.625361 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.639091 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] The token cache is being invalidate because project 44524486fb2641c7904cc6907bead070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81874) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:19.641483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:19.641483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:19.642564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1427/2841] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:19 2022] DELETE /identity/v3/projects/44524486fb2641c7904cc6907bead070 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:19.647957 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1415/2842] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:19.650291 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.650574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.653566 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:19.653719 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:19.672857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8074066-ebac-4112-9159-03639d52f689 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.677094 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfbd228-9e8e-44ea-a190-ba7987ba5075 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.679998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.680926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.681358 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.681813 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.682837 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.685380 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.685574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.686329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.686772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.687098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] PATH_INFO: `/v3/users/d0faa3e3e924487891ee9cb5e7fff35f` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.688768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0faa3e3e924487891ee9cb5e7fff35f)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.691469 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.704929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1416/2843] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 21:36:19.710226 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.723166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7f8fc-7b2f-4386-823d-f6361cf8d23f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:19.728476 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b69b51-c17a-41d2-96b8-c044afa8aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:19.729204 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69b51-c17a-41d2-96b8-c044afa8aee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.737232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69b51-c17a-41d2-96b8-c044afa8aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.737232 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b69b51-c17a-41d2-96b8-c044afa8aee1 admin admin] PATH_INFO: `/v3/projects/2113bed9d7c04c4691ff173ac0f3d75a/users/cce4a98431d74dd5a6494214d0d1188e/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.750858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b69b51-c17a-41d2-96b8-c044afa8aee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2113bed9d7c04c4691ff173ac0f3d75a, user_id=cce4a98431d74dd5a6494214d0d1188e, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:19.753658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b69b51-c17a-41d2-96b8-c044afa8aee1 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:19.853149 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1417/2844] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:19 2022] PUT /identity/v3/projects/2113bed9d7c04c4691ff173ac0f3d75a/users/cce4a98431d74dd5a6494214d0d1188e/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:36:19.858944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:19.859362 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:19.859759 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:19.881467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:19.882632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:19.950042 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1418/2845] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:19.952906 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0faa3e3e924487891ee9cb5e7fff35f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:19.961540 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:19.963211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:20.028638 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19152e8d-c257-4fae-8b5d-8920d18cfaee None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:20.032525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:20.033151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:20.033438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:20.033768 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:20.043780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:20.044048 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0faa3e3e924487891ee9cb5e7fff35f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:20.045624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:20.051127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:20.054241 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0faa3e3e924487891ee9cb5e7fff35f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:20.058996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1419/2846] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:20.071920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0faa3e3e924487891ee9cb5e7fff35f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:20.075712 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0faa3e3e924487891ee9cb5e7fff35f'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:20.082519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1428/2847] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:19 2022] DELETE /identity/v3/users/d0faa3e3e924487891ee9cb5e7fff35f => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:20.633506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:20.665068 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca425838-ac75-47c5-8ea3-ab7806d915bd service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:20.794238 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:20.796421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:20.796421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:20.796421 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] PATH_INFO: `/v3/projects/4a1bcdd3b4224cfc8c1f62f047367701` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:20.796996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1bcdd3b4224cfc8c1f62f047367701)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:20.799206 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:20.949501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1bcdd3b4224cfc8c1f62f047367701', 'request_id': 'req-58880ab0-0127-41a4-b360-0ec0e4836026'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:20.956399 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1bcdd3b4224cfc8c1f62f047367701', 'request_id': 'req-58880ab0-0127-41a4-b360-0ec0e4836026'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.067928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1bcdd3b4224cfc8c1f62f047367701', 'request_id': 'req-58880ab0-0127-41a4-b360-0ec0e4836026'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.080448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] The token cache is being invalidate because project 4a1bcdd3b4224cfc8c1f62f047367701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:21.080808 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.081549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-58880ab0-0127-41a4-b360-0ec0e4836026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.083079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1420/2848] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:20 2022] DELETE /identity/v3/projects/4a1bcdd3b4224cfc8c1f62f047367701 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:21.087835 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:21.090959 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:21.106466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:21.111746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbc307-b0a6-4398-83ad-c3d705fb629b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:21.222498 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:21.223834 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:21.224647 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:21.225340 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] PATH_INFO: `/v3/users/a3a48b445d3b4bf8a56e05569a910f91` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:21.227635 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a48b445d3b4bf8a56e05569a910f91)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:21.232103 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:21.547875 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a48b445d3b4bf8a56e05569a910f91'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.716464 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a48b445d3b4bf8a56e05569a910f91'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.732646 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a48b445d3b4bf8a56e05569a910f91'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.756971 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a48b445d3b4bf8a56e05569a910f91'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.758136 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a48b445d3b4bf8a56e05569a910f91'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:21.769351 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1429/2849] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:36:21 2022] DELETE /identity/v3/users/a3a48b445d3b4bf8a56e05569a910f91 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:21.893643 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:21.893975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:21.894952 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:21.949459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] MFA Rules not processed for user `cce4a98431d74dd5a6494214d0d1188e`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:21.952705 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:22.071140 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1421/2850] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:22.082934 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:22.084556 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:22.161800 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12a85e-365f-4120-b111-e8054ba21f95 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:22.167562 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998726-3bb0-489a-987b-5e5395e227ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:22.168510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998726-3bb0-489a-987b-5e5395e227ba service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:22.168798 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998726-3bb0-489a-987b-5e5395e227ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:22.169079 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998726-3bb0-489a-987b-5e5395e227ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:22.179230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998726-3bb0-489a-987b-5e5395e227ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:22.180814 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998726-3bb0-489a-987b-5e5395e227ba service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:22.194483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1430/2851] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:22.417061 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:22.419168 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:22.450016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:22.507438 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a979f-df2f-4a22-b81a-e32d2970aaa1 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:22.514986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:22.515652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:22.515972 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:22.516237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] PATH_INFO: `/v3/projects/9011c66879df4e0b8499aa92f31be283` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:22.517407 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9011c66879df4e0b8499aa92f31be283)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:22.519337 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:22.667803 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9011c66879df4e0b8499aa92f31be283', 'request_id': 'req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:22.677037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9011c66879df4e0b8499aa92f31be283', 'request_id': 'req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:22.766928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9011c66879df4e0b8499aa92f31be283', 'request_id': 'req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:22.780812 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] The token cache is being invalidate because project 9011c66879df4e0b8499aa92f31be283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:36:22.781102 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:22.781904 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-ee1e67e9-9a73-40ee-b18d-984b079ab0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:36:22.783447 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1422/2852] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:36:22 2022] DELETE /identity/v3/projects/9011c66879df4e0b8499aa92f31be283 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:22.788716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:22.789004 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:22.789293 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:22.811914 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:22.813225 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:22.836017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:22.882737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1431/2853] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:22.891003 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:22.891283 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:22.897334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:22.912856 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:22.914122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:22.923334 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1423/2854] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:22.934706 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:22.958131 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53232a97-73f7-4667-986a-88fba20d1d02 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:22.967240 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:22.968391 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:22.968905 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:22.969544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:22.970967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:36:22.971624 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:22.974583 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:22.999731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1432/2855] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:36:23.005126 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:23.024070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb7c8f-a1b4-4e3b-bf7d-33816c343785 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:23.034084 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:23.035299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.035842 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.036776 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.037800 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:23.041926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:23.228949 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:23.229663 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1424/2856] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:36:23 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:23.244518 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2d816-c6ec-4970-afb9-bb04d1327fd5 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:23.366149 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:23.367389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.367924 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.374176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.374176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:23.376544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:23.511831 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1433/2857] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:36:23 2022] POST /identity/v3/users => generated 286 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:36:23.518217 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:23.527628 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ebe4c-0ba2-4895-b3f3-6e47471bfe0f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:23.532966 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:23.533598 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.533919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.534213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.534872 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:23.536824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:23.549465 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1425/2858] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:23.554763 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:23.571020 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eab245-f86c-44a4-8bf4-f5045e9250cf admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:23.579192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070c194-a2aa-48e6-8543-b972cf0cf572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:23.580144 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c194-a2aa-48e6-8543-b972cf0cf572 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.580571 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c194-a2aa-48e6-8543-b972cf0cf572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.580980 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070c194-a2aa-48e6-8543-b972cf0cf572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.582003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c194-a2aa-48e6-8543-b972cf0cf572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:23.585106 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070c194-a2aa-48e6-8543-b972cf0cf572 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:23.603389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1434/2859] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:36:23.608176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:23.617787 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a818a2-890d-4da0-ab74-79e5f32e51a7 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:23.622858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:23.623501 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.623780 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.624092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] PATH_INFO: `/v3/projects/b233fa99a6f44783bdd72e14b28bf441/users/ee513b3ba1d545a696c08448370680e5/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.639626 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b233fa99a6f44783bdd72e14b28bf441, user_id=ee513b3ba1d545a696c08448370680e5, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:23.642410 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:23.722605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1426/2860] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:36:23 2022] PUT /identity/v3/projects/b233fa99a6f44783bdd72e14b28bf441/users/ee513b3ba1d545a696c08448370680e5/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:36:23.728532 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0671b-014b-4a7f-b3f3-9dc97f5455b9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.728869 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0671b-014b-4a7f-b3f3-9dc97f5455b9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.729187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0671b-014b-4a7f-b3f3-9dc97f5455b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.751649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-51b0671b-014b-4a7f-b3f3-9dc97f5455b9 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:23.752915 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-51b0671b-014b-4a7f-b3f3-9dc97f5455b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:23.814833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1435/2861] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:36:23.827689 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:23.829528 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:23.904182 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fae72c-564c-4276-8d76-9790c433f5dd admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:23.908092 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe45ef-0808-417a-acfc-e6322da64489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:23.908731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe45ef-0808-417a-acfc-e6322da64489 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:23.909004 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe45ef-0808-417a-acfc-e6322da64489 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:23.909282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe45ef-0808-417a-acfc-e6322da64489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:23.921914 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe45ef-0808-417a-acfc-e6322da64489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:23.923635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe45ef-0808-417a-acfc-e6322da64489 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:23.937169 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1427/2862] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:24.299094 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:24.332237 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:24.332942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:24.332942 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:24.360274 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] MFA Rules not processed for user `cce4a98431d74dd5a6494214d0d1188e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:24.361868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:24.426681 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1436/2863] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:24.543242 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:25.887414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485e619-9d02-4ad0-89cc-c7167af2a77e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:25.888103 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485e619-9d02-4ad0-89cc-c7167af2a77e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:25.889237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485e619-9d02-4ad0-89cc-c7167af2a77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:25.936580 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-7485e619-9d02-4ad0-89cc-c7167af2a77e None None] MFA Rules not processed for user `ee513b3ba1d545a696c08448370680e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:25.939611 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7485e619-9d02-4ad0-89cc-c7167af2a77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:26.057793 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1428/2864] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:26.071566 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:26.073482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:26.158682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d352964-a890-470d-b56d-022eb3a47c19 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:26.162939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:26.163574 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:26.163866 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:26.164174 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:26.175683 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:26.177947 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:26.188846 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1437/2865] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:36:28.257836 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:28.263474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:28.263474 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:28.290567 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] MFA Rules not processed for user `ee513b3ba1d545a696c08448370680e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:28.292513 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:28.300002 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1429/2866] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:36:30.062003 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:30.072401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6a00d-d511-4408-8bb8-25bcc2aa7ba9 service cinder] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:30.076192 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:30.076847 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:30.077152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:30.077401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:30.079189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:30.160017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:30.165459 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:30.183244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1438/2867] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:34.006331 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:34.017831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77c6e2-e803-4f98-a8ee-cb53e90a2644 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:34.022011 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:34.022591 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:34.022873 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:34.023151 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:34.025732 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:34.122853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:34.124496 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5801f3-9b9b-40d3-ae98-36848e747612 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:34.140985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1430/2868] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:36.692050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:36:36.695829 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:36.709317 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b136296-42ed-439b-8142-d8ca1c9c530d service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:36:36.713555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:36:36.714188 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:36.714462 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:36.714744 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:36.716633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:36.733952 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:36:36.735408 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:36:36.757252 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1439/2869] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:36:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:36:44.135384 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=74 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:46.744011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=72 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:36:50.412851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb5123-7419-46a8-ad13-3adab112597e None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:36:50.413462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb5123-7419-46a8-ad13-3adab112597e None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:36:50.413633 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb5123-7419-46a8-ad13-3adab112597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:36:50.433095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0ceb5123-7419-46a8-ad13-3adab112597e None None] MFA Rules not processed for user `ee513b3ba1d545a696c08448370680e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:36:50.434350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0ceb5123-7419-46a8-ad13-3adab112597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:36:50.441368 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1431/2870] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:36:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:00.427397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:01.560428 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:01.569677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0798bdc-8479-46dd-9d48-e4f43d425448 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:01.573761 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:01.574384 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:01.574663 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:01.574940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:01.576794 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:01.586817 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:01.588306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:01.598137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1440/2871] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:11.594031 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=6 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:18.744599 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:18.745166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:18.745166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:18.801220 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:18.802464 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:18.872726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1432/2872] 10.222.0.65 () {58 vars in 1009 bytes} [Mon Aug 29 21:37:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:19.033322 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:19.035382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:19.137442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eda95d-dc70-46ee-aafa-ebb857e65d0b service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:19.143005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71761c1b-7560-4728-991e-7a0f5bfd8434 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:19.143743 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71761c1b-7560-4728-991e-7a0f5bfd8434 service swift] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:19.144058 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71761c1b-7560-4728-991e-7a0f5bfd8434 service swift] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:19.144437 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71761c1b-7560-4728-991e-7a0f5bfd8434 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:19.154472 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71761c1b-7560-4728-991e-7a0f5bfd8434 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:19.156153 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71761c1b-7560-4728-991e-7a0f5bfd8434 service swift] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:19.166682 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1441/2873] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:37:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:37:28.869925 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=40 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:29.162582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=41 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:38.916587 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:38.918571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:38.943656 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a31eb-1bb8-4cb3-ba90-78957b21b379 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:38.954859 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:38.955940 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:38.956435 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:38.956967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:38.960100 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:38.984382 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:38.986186 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:39.002975 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1433/2874] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:37:40.165954 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:40.166501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:40.166501 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:40.188684 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] MFA Rules not processed for user `3c2842cc986347dc802236ee27b9de50`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:40.189928 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:40.197939 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1442/2875] 10.222.0.65 () {58 vars in 967 bytes} [Mon Aug 29 21:37:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:40.215411 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:40.225181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451924d-a7a1-4c73-bf86-5e28a34861ba service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:40.230455 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb437e7c06504d0aa7891269ed02f9e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:40.231083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:40.231365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:40.231640 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:40.241928 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:40.243602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:40.253448 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1434/2876] 10.222.0.65 () {60 vars in 1384 bytes} [Mon Aug 29 21:37:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 21:37:45.920039 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:45.934287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987b4cd-e1c4-4ac6-8fe3-57845b65e520 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:45.939510 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:45.940208 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:45.940495 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:45.940767 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] PATH_INFO: `/v3/users/ee513b3ba1d545a696c08448370680e5` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:45.941969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee513b3ba1d545a696c08448370680e5)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:45.944119 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:46.177251 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee513b3ba1d545a696c08448370680e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:46.428772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee513b3ba1d545a696c08448370680e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:46.437005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee513b3ba1d545a696c08448370680e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:46.450330 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee513b3ba1d545a696c08448370680e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:46.451068 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-13300648-a662-45c6-a0b7-8419db2d0678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee513b3ba1d545a696c08448370680e5'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:46.457387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1443/2877] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:37:45 2022] DELETE /identity/v3/users/ee513b3ba1d545a696c08448370680e5 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:46.746722 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:46.761956 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66fa6e-fb90-4035-82db-c27489b1a8a1 service swift] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:46.843635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:46.844350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:46.844846 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:46.845120 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] PATH_INFO: `/v3/projects/b233fa99a6f44783bdd72e14b28bf441` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:46.846310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b233fa99a6f44783bdd72e14b28bf441)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:46.848271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:46.967243 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b233fa99a6f44783bdd72e14b28bf441', 'request_id': 'req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:46.975725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b233fa99a6f44783bdd72e14b28bf441', 'request_id': 'req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:47.068537 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b233fa99a6f44783bdd72e14b28bf441', 'request_id': 'req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:47.081903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] The token cache is being invalidate because project b233fa99a6f44783bdd72e14b28bf441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:37:47.082250 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:47.082980 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:37:47.085318 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1435/2878] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:37:46 2022] DELETE /identity/v3/projects/b233fa99a6f44783bdd72e14b28bf441 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:37:51.409858 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:51.410449 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:51.410853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:51.433376 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] MFA Rules not processed for user `cce4a98431d74dd5a6494214d0d1188e`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:37:51.434605 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:51.497755 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1444/2879] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:37:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:37:51.747863 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:51.749404 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:51.812613 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3cda-d9c0-462d-80aa-ae0e6191cea3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:51.816571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:51.817181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:51.817461 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:51.817731 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:51.820135 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:51.832941 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:51.835037 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:51.845344 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1436/2880] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:37:52.336678 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:37:52.357582 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:52.422497 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906031-91a1-4925-8371-1e7ab6f0e954 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:37:52.427982 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:37:52.428652 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:37:52.428934 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:37:52.429211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:37:52.431009 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:37:52.444632 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:37:52.446294 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:37:52.453147 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:37:52.457964 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1445/2881] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:37:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:38:01.840430 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:03.656588 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:38:03.659207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:03.789719 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908d685-6588-43b2-b606-a9cca6b44c93 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:38:03.796922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:38:03.797563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:03.797840 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:03.798121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:03.800310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:03.816883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:38:03.819549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:38:03.832051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1437/2882] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:38:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:38:13.827028 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=34 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:38:59.441868 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:38:59.452519 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2bb0c-aff6-4b6b-bcae-eeb04c8bd744 service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:38:59.456209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:38:59.456856 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:59.457137 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:59.457410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:59.459718 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:59.470797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:38:59.472287 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:38:59.481423 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1446/2883] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:38:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:38:59.561142 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:38:59.572390 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ae427-9da1-49dd-af35-bba65d4cab9e service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:38:59.576412 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:38:59.577080 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:38:59.577479 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:38:59.577764 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:38:59.579594 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:38:59.594121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:38:59.595820 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:38:59.607156 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1438/2884] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:38:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:03.440597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:03.444853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:03.533736 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051c33-79d4-4678-a321-4167f6a85cb3 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:03.541350 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:03.542025 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:03.542307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:03.542586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] PATH_INFO: `/v3/users/cce4a98431d74dd5a6494214d0d1188e` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:03.543933 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce4a98431d74dd5a6494214d0d1188e)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:03.546117 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:03.737105 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce4a98431d74dd5a6494214d0d1188e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:03.828857 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce4a98431d74dd5a6494214d0d1188e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:03.839442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce4a98431d74dd5a6494214d0d1188e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:03.854329 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce4a98431d74dd5a6494214d0d1188e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:03.855289 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9a5fb506-1687-4f29-abea-c751289bad6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce4a98431d74dd5a6494214d0d1188e'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:03.862656 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1447/2885] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:03 2022] DELETE /identity/v3/users/cce4a98431d74dd5a6494214d0d1188e => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:04.152788 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:04.168016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091889df-7435-488b-bda3-176f0e3a9eba service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:04.253218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:04.253870 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.254157 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.254436 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] PATH_INFO: `/v3/projects/2113bed9d7c04c4691ff173ac0f3d75a` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.256749 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2113bed9d7c04c4691ff173ac0f3d75a)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:04.258654 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:04.380821 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2113bed9d7c04c4691ff173ac0f3d75a', 'request_id': 'req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:04.387726 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2113bed9d7c04c4691ff173ac0f3d75a', 'request_id': 'req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:04.489903 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2113bed9d7c04c4691ff173ac0f3d75a', 'request_id': 'req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:04.501680 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] The token cache is being invalidate because project 2113bed9d7c04c4691ff173ac0f3d75a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:04.501963 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:04.502615 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-dabaa3b3-153a-47cb-970f-49ba4a98fdfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:04.504147 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1439/2886] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:04 2022] DELETE /identity/v3/projects/2113bed9d7c04c4691ff173ac0f3d75a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:04.509267 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.509764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.509764 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.530211 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:04.531434 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:04.601082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1448/2887] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:39:04.609059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.609346 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.609622 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.631345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:04.632586 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:04.640600 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1440/2888] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:04.645246 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:04.655853 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe8399-0fd3-42dd-add0-5ec4d91d2720 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:04.660555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:04.661177 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.661461 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.662352 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.662999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:04.663306 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:04.665000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:04.666580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:04.677221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1449/2889] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 21:39:04.682499 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:04.694207 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d4eed-1a13-41ef-ab46-cb89c4474c10 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:04.703693 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:04.704321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.704605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.704919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.705541 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:04.707459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:04.865121 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1441/2890] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:04 2022] POST /identity/v3/projects => generated 397 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:04.869450 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:04.880171 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9147-b87c-4915-92cb-4022381b5234 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:04.944521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:04.944842 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:04.945141 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:04.945401 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:04.946096 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:04.948067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:05.057657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1450/2891] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:04 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:05.060088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:05.071857 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b394-3171-460c-80c6-98979be29cbc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:05.076684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:05.077322 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:05.077620 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:05.077922 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:05.078671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:05.080482 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:05.092605 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1442/2892] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:05.097731 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:05.112505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e483f-66ec-4b9f-af2f-01fda869f354 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:05.117720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b746b3-ac1b-4047-a4b9-9521bdd37f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:05.118346 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b746b3-ac1b-4047-a4b9-9521bdd37f57 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:05.118629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b746b3-ac1b-4047-a4b9-9521bdd37f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:05.118902 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b746b3-ac1b-4047-a4b9-9521bdd37f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:05.119555 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b746b3-ac1b-4047-a4b9-9521bdd37f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:05.121230 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b746b3-ac1b-4047-a4b9-9521bdd37f57 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:05.142011 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1451/2893] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:05.147819 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:05.159046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc6f24-389d-402b-813c-220f25686b4e admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:05.164131 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:05.165050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:05.165335 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:05.166272 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] PATH_INFO: `/v3/projects/79d34bbc321d4e6cb255494db8d70d27/users/a6a0362f06d546da885d94e6dc41c2de/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:05.182420 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d34bbc321d4e6cb255494db8d70d27, user_id=a6a0362f06d546da885d94e6dc41c2de, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:05.185216 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:05.267434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1443/2894] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:05 2022] PUT /identity/v3/projects/79d34bbc321d4e6cb255494db8d70d27/users/a6a0362f06d546da885d94e6dc41c2de/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:39:05.288655 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:05.288940 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:05.289219 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:05.319347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] MFA Rules not processed for user `a6a0362f06d546da885d94e6dc41c2de`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:05.320597 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:05.390047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1452/2895] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:05.405047 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:05.406921 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:05.418330 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:05.486279 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b972b37-3531-4ecc-8fba-9a077b86549a admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:05.490637 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8346df8-0e59-490c-9797-0ea0ec4be137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3fb180119c3a4f07933f7d24f4d22121', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:05.491283 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8346df8-0e59-490c-9797-0ea0ec4be137 service cinder] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:05.491563 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8346df8-0e59-490c-9797-0ea0ec4be137 service cinder] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:05.491834 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8346df8-0e59-490c-9797-0ea0ec4be137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:05.504890 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8346df8-0e59-490c-9797-0ea0ec4be137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:05.506932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8346df8-0e59-490c-9797-0ea0ec4be137 service cinder] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:05.517310 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1444/2896] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:05 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:14.818236 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:14.886768 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85ceca-b0fd-4fac-9c8b-cfb2430af29d None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:14.891593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:14.892263 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:14.892544 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:14.892823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] PATH_INFO: `/v3/users/a6a0362f06d546da885d94e6dc41c2de` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:14.894042 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a0362f06d546da885d94e6dc41c2de)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:14.896573 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:14.899311 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:15.173710 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a0362f06d546da885d94e6dc41c2de'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:15.245347 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a0362f06d546da885d94e6dc41c2de'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:15.253181 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a0362f06d546da885d94e6dc41c2de'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:15.265926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a0362f06d546da885d94e6dc41c2de'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:15.266653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a0362f06d546da885d94e6dc41c2de'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:15.274012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1453/2897] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:39:14 2022] DELETE /identity/v3/users/a6a0362f06d546da885d94e6dc41c2de => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:15.279095 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:15.279565 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:15.279664 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:15.298946 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:15.300179 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:15.362998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1445/2898] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:15.374032 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:15.375734 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:15.436090 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289821af-a7c1-4abf-8f52-09933ab1234e admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:15.439818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550042f-8a1b-4647-9957-71244d32660c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:15.440456 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550042f-8a1b-4647-9957-71244d32660c service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:15.440733 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550042f-8a1b-4647-9957-71244d32660c service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:15.441002 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550042f-8a1b-4647-9957-71244d32660c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:15.451341 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550042f-8a1b-4647-9957-71244d32660c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:15.452816 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550042f-8a1b-4647-9957-71244d32660c service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:15.463145 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1454/2899] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:16.030007 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.032510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:16.046624 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf7a4c-1691-42f9-8506-fd4765f4d1d6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.053744 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.054905 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.055409 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.055794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] PATH_INFO: `/v3/projects/79d34bbc321d4e6cb255494db8d70d27` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.057960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d34bbc321d4e6cb255494db8d70d27)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:16.060855 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:16.239824 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d34bbc321d4e6cb255494db8d70d27', 'request_id': 'req-3672eb43-5f1f-4af2-99c6-e72fe02363e3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:16.248687 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d34bbc321d4e6cb255494db8d70d27', 'request_id': 'req-3672eb43-5f1f-4af2-99c6-e72fe02363e3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:16.369974 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d34bbc321d4e6cb255494db8d70d27', 'request_id': 'req-3672eb43-5f1f-4af2-99c6-e72fe02363e3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:16.382770 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] The token cache is being invalidate because project 79d34bbc321d4e6cb255494db8d70d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:39:16.383057 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:16.383727 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-3672eb43-5f1f-4af2-99c6-e72fe02363e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:39:16.385364 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1446/2900] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:39:16 2022] DELETE /identity/v3/projects/79d34bbc321d4e6cb255494db8d70d27 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:16.390575 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.391082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.391191 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.411664 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:16.412920 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:16.471493 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1455/2901] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:16.480258 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.480549 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.480854 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.502713 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:16.506432 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:16.513671 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1447/2902] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:16.518382 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.530074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf089ad-eb07-4e5b-865f-4e85141cbfaf None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.537504 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.538152 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.538431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.538707 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.539364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81874) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 21:39:16.539667 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:16.541356 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:16.554577 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1456/2903] 10.222.0.65 () {58 vars in 1153 bytes} [Mon Aug 29 21:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 21:39:16.559029 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.568434 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f636c90-548e-499d-abb4-9f04532dfed5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.573181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.573815 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.573998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.574269 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.574934 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:16.576944 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:16.675960 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1448/2904] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:16 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:16.680406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.690593 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dd16f-8df1-490f-8ad6-b08f3a089f93 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.698969 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:16.753503 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.754114 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.754394 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.754669 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.755308 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:16.757305 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:16.893325 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1457/2905] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:16 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:16.898064 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.903785 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:16.909321 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5bbe-aa1e-4e70-8046-feecac7f8e14 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.914517 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.915079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.915357 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.915630 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.916320 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:16.917937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:16.928847 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1449/2906] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:16.933442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.944161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac152d3-8d20-43d4-ba2c-6be8a5ab5c2b admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.948804 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed5d24-3463-4eb9-8e96-a393dcd80e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.949410 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed5d24-3463-4eb9-8e96-a393dcd80e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.949689 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed5d24-3463-4eb9-8e96-a393dcd80e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.949967 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed5d24-3463-4eb9-8e96-a393dcd80e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:16.950610 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed5d24-3463-4eb9-8e96-a393dcd80e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:16.952244 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed5d24-3463-4eb9-8e96-a393dcd80e05 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:16.963711 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1458/2907] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:16.968228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:16.979281 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77feaeb4-3827-406f-b908-6891e755069f admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:16.983828 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:16.984550 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:16.984858 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:16.985138 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] PATH_INFO: `/v3/projects/77472ffae4da491881e41c08aa09e4fc/users/0d57f52dc9044a259e6acbde81ac0476/roles/87804c58f0314370a20ec138295f6863` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:17.006116 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77472ffae4da491881e41c08aa09e4fc, user_id=0d57f52dc9044a259e6acbde81ac0476, role_id=87804c58f0314370a20ec138295f6863)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:17.009146 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:17.140089 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1450/2908] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:16 2022] PUT /identity/v3/projects/77472ffae4da491881e41c08aa09e4fc/users/0d57f52dc9044a259e6acbde81ac0476/roles/87804c58f0314370a20ec138295f6863 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:17.145677 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77958-26c3-4663-9d2a-dce29fdc014a None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:17.145956 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77958-26c3-4663-9d2a-dce29fdc014a None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:17.146649 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77958-26c3-4663-9d2a-dce29fdc014a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:17.167442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-62e77958-26c3-4663-9d2a-dce29fdc014a None None] MFA Rules not processed for user `6fd5eb0c7a1f4bdbb078438d56380575`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:17.168975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-62e77958-26c3-4663-9d2a-dce29fdc014a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:17.240484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1459/2909] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:17.254603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:17.258117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:17.334150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7604a5-ce2b-4a8a-be9b-1a75e4d794b2 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:17.338706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:17.339136 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:17.339309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:17.339548 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:17.351602 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:17.353271 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:17.367339 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1451/2910] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:23.877307 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:23.877957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:23.877957 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:23.909818 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] MFA Rules not processed for user `0d57f52dc9044a259e6acbde81ac0476`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:23.911590 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:23.984981 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1460/2911] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:39:23.990488 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:24.001038 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d21789-4092-4707-a068-c1b22cf9468a service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:24.006016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:24.006649 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:24.006930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:24.007218 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:24.007868 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:24.009961 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:24.110778 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1452/2912] 10.222.0.65 () {60 vars in 1153 bytes} [Mon Aug 29 21:39:23 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:24.115729 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:24.127793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621360ed-1986-406d-a8eb-fbbfad0dd5f9 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:24.148209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:24.215000 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:24.215999 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:24.216448 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:24.216823 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] PATH_INFO: `/v3/users` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:24.220674 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:24.223276 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:24.385235 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1461/2913] 10.222.0.65 () {60 vars in 1144 bytes} [Mon Aug 29 21:39:24 2022] POST /identity/v3/users => generated 295 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 21:39:24.390747 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:24.402875 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720b5f7-be8b-4059-9eac-e6ed6a1464b0 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:24.411494 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:24.412085 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:24.412585 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:24.412725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:24.413546 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:24.415044 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:24.441314 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1453/2914] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:24.445992 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:24.457053 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a79e9-65ca-4894-b478-8ee134b41c68 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:24.462453 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:24.463166 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:24.463442 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:24.463720 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] PATH_INFO: `/v3/projects/7644cad195174da080a9989cec76033d/users/341de9d4ba86482f890e1c386dc22594/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:24.489912 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7644cad195174da080a9989cec76033d, user_id=341de9d4ba86482f890e1c386dc22594, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:24.492824 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:24.579585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1462/2915] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:24 2022] PUT /identity/v3/projects/7644cad195174da080a9989cec76033d/users/341de9d4ba86482f890e1c386dc22594/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:24.584511 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:24.596266 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a7d87-7467-4fbd-9690-2b11d8c35b46 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:24.662937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56846e8-153f-4b89-bbd6-c0df1c1704ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:24.664822 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56846e8-153f-4b89-bbd6-c0df1c1704ca admin admin] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:24.665130 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56846e8-153f-4b89-bbd6-c0df1c1704ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:24.665406 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56846e8-153f-4b89-bbd6-c0df1c1704ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:24.666104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56846e8-153f-4b89-bbd6-c0df1c1704ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:24.667782 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56846e8-153f-4b89-bbd6-c0df1c1704ca admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:24.678991 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1454/2916] 10.222.0.65 () {58 vars in 1122 bytes} [Mon Aug 29 21:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 21:39:24.683807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:24.696441 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c7ac1-af4b-44b2-8dcd-694dc6f5c241 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:24.704067 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:24.705620 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:24.706406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:24.706771 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] PATH_INFO: `/v3/projects/7644cad195174da080a9989cec76033d/users/341de9d4ba86482f890e1c386dc22594/roles/d8cbc03a891c4dbfb863c52fbeaddbc3` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:24.710631 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7644cad195174da080a9989cec76033d, user_id=341de9d4ba86482f890e1c386dc22594, role_id=d8cbc03a891c4dbfb863c52fbeaddbc3)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:24.713724 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:24.733838 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1463/2917] 10.222.0.65 () {60 vars in 1483 bytes} [Mon Aug 29 21:39:24 2022] PUT /identity/v3/projects/7644cad195174da080a9989cec76033d/users/341de9d4ba86482f890e1c386dc22594/roles/d8cbc03a891c4dbfb863c52fbeaddbc3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:39:31.642112 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.643016 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.643287 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:31.687632 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] MFA Rules not processed for user `341de9d4ba86482f890e1c386dc22594`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:31.690018 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:31.723750 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:31.792289 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1455/2918] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 21:39:31.808614 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:31.811466 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:31.934560 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0c7c5-c526-4ec0-9684-f88ae0615dc6 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:31.940979 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73069aa-e2a3-4a7b-9d06-34b7921bd5f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:31.941860 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73069aa-e2a3-4a7b-9d06-34b7921bd5f4 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:31.942815 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73069aa-e2a3-4a7b-9d06-34b7921bd5f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:31.943156 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73069aa-e2a3-4a7b-9d06-34b7921bd5f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:32.043680 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73069aa-e2a3-4a7b-9d06-34b7921bd5f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:32.045255 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73069aa-e2a3-4a7b-9d06-34b7921bd5f4 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:32.057082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1464/2919] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:39:41.804345 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=19 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:42.050785 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:39:43.240635 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:39:43.242458 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:43.341883 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4ce9-31c6-45a4-a612-d8f2982207b6 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:39:43.349052 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:39:43.349761 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:43.350088 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:43.350440 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:43.353530 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:43.370177 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:39:43.373524 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:39:43.393165 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1456/2920] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:39:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:39:53.348528 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:39:53.351268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:39:53.351268 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:39:53.369353 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] MFA Rules not processed for user `0d57f52dc9044a259e6acbde81ac0476`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:39:53.370580 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:39:53.377657 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1465/2921] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:39:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:39:53.386307 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=32 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:03.363389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=3 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:12.794629 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:12.796073 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:12.861466 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b1cb2-e4f5-4a81-aae7-96e6d345c847 service placement] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:12.865676 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:12.866297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:12.866574 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:12.866851 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:12.876579 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:12.879328 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:12.891282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1457/2922] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 21:40:22.886525 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=26 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:43.548659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:43.570765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7f1ea-d26c-4228-9ee2-d0e532ea4b38 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:43.576861 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:43.577483 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:43.577766 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:43.578041 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:43.580026 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:43.641129 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:43.658975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:43.658975 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1466/2923] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:44.068996 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:44.078015 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c42d-6844-4268-af4f-c926ac400627 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:44.081700 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:44.082336 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:44.082666 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:44.082894 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:44.084755 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:44.151366 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:44.152850 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:44.163210 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1458/2924] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:47.928320 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:47.929772 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:47.947054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619be76-dcd5-4f4c-bbb4-b003975ed199 service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:47.953355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:47.953995 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:47.954300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:47.954747 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:47.958592 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:47.973594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:47.976633 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:47.995765 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1467/2925] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:52.026079 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:52.040523 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390d526-fce7-406c-97f2-4dcd61a891cb service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:52.045641 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:52.046510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:52.046510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:52.046781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:52.048864 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:52.061484 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:52.063071 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:52.073459 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1459/2926] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:40:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:40:52.223833 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:52.225889 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:52.248752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffd43f-f00d-4309-ad00-094dd5605aa1 service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:52.248752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:52.248752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:52.248752 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:52.249283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] PATH_INFO: `/v3/users/6576ed5b370049f5bd091512d9803d3c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:52.249283 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6576ed5b370049f5bd091512d9803d3c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:52.249716 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:52.503118 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6576ed5b370049f5bd091512d9803d3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:52.609327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6576ed5b370049f5bd091512d9803d3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:52.617506 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6576ed5b370049f5bd091512d9803d3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:52.630714 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6576ed5b370049f5bd091512d9803d3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:52.631482 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6576ed5b370049f5bd091512d9803d3c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:52.637615 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1468/2927] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:40:52 2022] DELETE /identity/v3/users/6576ed5b370049f5bd091512d9803d3c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:40:52.942967 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:52.954446 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef51813d-9ab8-4daa-9787-a4cc7477c8e9 service placement] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:53.032397 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:53.034725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:53.035166 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:53.035597 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] PATH_INFO: `/v3/projects/ecd13ced46854940b3765ee641390212` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:53.036970 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd13ced46854940b3765ee641390212)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:53.038919 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:53.158691 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd13ced46854940b3765ee641390212', 'request_id': 'req-b628e057-8270-4374-9805-ffa6650b3be3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.165886 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd13ced46854940b3765ee641390212', 'request_id': 'req-b628e057-8270-4374-9805-ffa6650b3be3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.242403 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd13ced46854940b3765ee641390212', 'request_id': 'req-b628e057-8270-4374-9805-ffa6650b3be3'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.263230 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] The token cache is being invalidate because project ecd13ced46854940b3765ee641390212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:40:53.263539 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.264506 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.266104 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1460/2928] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:40:52 2022] DELETE /identity/v3/projects/ecd13ced46854940b3765ee641390212 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 21:40:53.270585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:53.272616 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:53.287209 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6c101-6946-4521-a665-ad2ee2284408 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:53.348161 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:53.357559 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:53.359098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:53.359098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:53.359354 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] PATH_INFO: `/v3/users/80a644b6ea534e75a529f5514a9be68c` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:53.361943 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a644b6ea534e75a529f5514a9be68c)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:53.365505 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:53.601050 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a644b6ea534e75a529f5514a9be68c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.684046 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a644b6ea534e75a529f5514a9be68c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.694054 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a644b6ea534e75a529f5514a9be68c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.707431 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a644b6ea534e75a529f5514a9be68c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.708512 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a644b6ea534e75a529f5514a9be68c'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:53.715585 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1469/2929] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:40:53 2022] DELETE /identity/v3/users/80a644b6ea534e75a529f5514a9be68c => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:40:53.996684 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:40:53.998245 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:40:54.018781 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628e057-8270-4374-9805-ffa6650b3be3 admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:40:54.039845 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:40:54.118228 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:40:54.119020 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:40:54.119332 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:40:54.119658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] PATH_INFO: `/v3/projects/41023f7417314c0b9e63e0d68bbb7d2f` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:40:54.121107 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=41023f7417314c0b9e63e0d68bbb7d2f)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:40:54.123192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:40:54.287918 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41023f7417314c0b9e63e0d68bbb7d2f', 'request_id': 'req-46ade696-4442-4b8c-b631-803a39a9d0fc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.295046 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41023f7417314c0b9e63e0d68bbb7d2f', 'request_id': 'req-46ade696-4442-4b8c-b631-803a39a9d0fc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.357860 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41023f7417314c0b9e63e0d68bbb7d2f', 'request_id': 'req-46ade696-4442-4b8c-b631-803a39a9d0fc'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.373203 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] The token cache is being invalidate because project 41023f7417314c0b9e63e0d68bbb7d2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:40:54.376355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.376355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:40:54.376355 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1461/2930] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:40:53 2022] DELETE /identity/v3/projects/41023f7417314c0b9e63e0d68bbb7d2f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:41:03.711499 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:41:04.368835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:13.143591 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:13.145239 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:13.211302 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686d744-c8ca-4661-9ec6-68113c5b2bef admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:13.261281 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:13.261893 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:13.262176 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:13.262457 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:13.264994 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:13.340584 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:13.342098 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d68c5-dc03-498f-9876-11b271d8ecb1 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:13.354075 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1470/2931] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:14.549986 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:14.551467 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.564176 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ade696-4442-4b8c-b631-803a39a9d0fc admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:14.568063 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:14.568659 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:14.568938 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:14.569213 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:14.570866 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:14.584725 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:14.586150 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:14.597211 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1462/2932] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:16.945878 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] REQUEST_METHOD: `POST` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:16.946393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:16.946507 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:16.970095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.auth.core [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] MFA Rules not processed for user `0d57f52dc9044a259e6acbde81ac0476`. Rule list: `[]` (Enabled: `True`). {{(pid=81874) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:42:16.971314 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:17.027012 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1471/2933] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:42:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:42:17.349712 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:17.359510 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abba2b-8ef9-4f14-be07-f1d39f109dc6 service neutron] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:17.363059 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:17.363658 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:17.363976 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:17.364251 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:17.365926 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:17.375835 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:17.377290 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de88ca-fa7a-49bd-a7ea-77be93fedaa9 service neutron] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:17.385297 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1463/2934] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:21.127978 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:42:21.134557 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=100 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:21.145033 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28194-36f5-478e-a014-95d38a0b0a84 None None] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:42:21.150393 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:42:21.151481 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:42:21.152014 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:42:21.152508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:42:21.155424 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:42:21.171688 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:42:21.173229 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:42:21.182629 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1472/2935] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:42:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:42:27.381861 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=12 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:42:31.213387 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=5 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:12.771117 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] REQUEST_METHOD: `POST` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:12.771973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:12.771973 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:12.790852 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.auth.core [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] MFA Rules not processed for user `0d57f52dc9044a259e6acbde81ac0476`. Rule list: `[]` (Enabled: `True`). {{(pid=81875) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 21:44:12.792083 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:12.798998 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1464/2936] 10.222.0.65 () {58 vars in 958 bytes} [Mon Aug 29 21:44:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 21:44:14.850279 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:14.851715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:14.928355 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71b5ee-1eb9-437b-8e66-9277aa051548 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:14.933327 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:14.933937 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:14.934214 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:14.934485 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:14.936421 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:14.949400 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:14.951021 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:14.961473 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1473/2937] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:21.700288 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:21.701805 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:21.718428 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d59efb-e29e-4520-a0aa-8a95d4e6f9f5 None None] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:21.724401 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:21.725017 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:21.725300 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:21.725571 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:21.727365 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:21.741353 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:21.742977 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:21.754930 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1465/2938] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:24.957018 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=32 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:31.750014 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=9 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:32.169269 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:32.179187 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdcdf8-e2e4-4086-8805-92af2cc342bc service placement] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:32.182702 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:32.183793 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:32.184095 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:32.184364 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:32.186221 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:32.197013 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:32.198443 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:32.208109 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1474/2939] 10.222.0.65 () {60 vars in 1396 bytes} [Mon Aug 29 21:44:32 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:33.558806 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:33.561462 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:33.575197 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847ae1d-5417-48ab-b28c-516c7d494828 service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:33.581024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:33.581743 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:33.582024 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:33.582299 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:33.584122 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:33.596603 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:33.598202 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:33.609192 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1466/2940] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:38.065540 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:38.078029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03f0c7-24e5-4d2c-ac42-de368cfe8dc8 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:38.083659 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5014eab4c7479f9ee493e31c91b3f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:38.084371 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:38.084715 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:38.085074 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:38.087029 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:38.100843 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:38.102549 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:38.114617 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1475/2941] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:43.604706 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=6 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:47.171652 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:44:47.173235 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:47.249867 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c56b4-cbf8-4c26-bd96-e678592094a3 service nova] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:44:47.255309 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:44:47.256937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:44:47.256937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:44:47.256937 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:44:47.258190 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:44:47.270932 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:44:47.272798 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:44:47.284282 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1467/2942] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:44:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:44:48.109899 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=12 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:44:57.279929 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=32 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:24.978429 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:24.979864 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:25.043594 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81fe8b-4b06-43f7-85e8-8da2152be3e7 service nova] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:25.047277 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '73aebcec5ecc40ea9897fc7450c1cd0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:25.047927 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:25.048245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:25.048524 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:25.050284 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:25.136487 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:25.137945 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:25.156508 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1476/2943] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:46:28.690373 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:28.716387 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa8be3-333b-4444-81e2-0b5aafd2eb86 service placement] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:28.721556 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:28.722237 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] REQUEST_METHOD: `GET` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:28.722518 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:28.722792 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:28.725414 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:28.737994 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:28.739661 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:28.750070 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1468/2944] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:46:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:46:32.376089 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:32.381139 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:32.390613 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a7ebe-16ec-4923-96b5-76bc91ac49d7 service neutron] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:32.395807 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:32.396602 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:32.397005 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:32.397285 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] PATH_INFO: `/v3/users/0d57f52dc9044a259e6acbde81ac0476` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:32.399586 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d57f52dc9044a259e6acbde81ac0476)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:32.401564 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:32.624083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d57f52dc9044a259e6acbde81ac0476'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:32.706770 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d57f52dc9044a259e6acbde81ac0476'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:32.714797 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d57f52dc9044a259e6acbde81ac0476'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:32.721746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:32.727430 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d57f52dc9044a259e6acbde81ac0476'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:32.728202 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d57f52dc9044a259e6acbde81ac0476'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:32.734338 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1477/2945] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:46:32 2022] DELETE /identity/v3/users/0d57f52dc9044a259e6acbde81ac0476 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:32.962127 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:32.977915 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc726710-d1bd-4c6b-9a63-11b8fd22816c service placement] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:33.069152 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:33.069794 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:33.070076 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:33.070350 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] PATH_INFO: `/v3/projects/77472ffae4da491881e41c08aa09e4fc` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:33.071553 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77472ffae4da491881e41c08aa09e4fc)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:33.073545 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:33.199051 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77472ffae4da491881e41c08aa09e4fc', 'request_id': 'req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.206304 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77472ffae4da491881e41c08aa09e4fc', 'request_id': 'req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.301093 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77472ffae4da491881e41c08aa09e4fc', 'request_id': 'req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.312900 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] The token cache is being invalidate because project 77472ffae4da491881e41c08aa09e4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:46:33.317896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.317896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.317896 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1469/2946] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:46:32 2022] DELETE /identity/v3/projects/77472ffae4da491881e41c08aa09e4fc => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:33.319484 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:33.320916 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:33.334037 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4f20f-c3b1-44ba-950f-6d4212c5c8ec admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:33.391300 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:33.392047 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:33.392245 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:33.392520 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] PATH_INFO: `/v3/users/341de9d4ba86482f890e1c386dc22594` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:33.393660 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=341de9d4ba86482f890e1c386dc22594)` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:33.395653 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:33.680746 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '341de9d4ba86482f890e1c386dc22594'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.780926 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '341de9d4ba86482f890e1c386dc22594'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.789055 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '341de9d4ba86482f890e1c386dc22594'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.801082 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '341de9d4ba86482f890e1c386dc22594'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.801698 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.notifications [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '341de9d4ba86482f890e1c386dc22594'} {{(pid=81874) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:33.807791 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1478/2947] 10.222.0.65 () {58 vars in 1224 bytes} [Mon Aug 29 21:46:33 2022] DELETE /identity/v3/users/341de9d4ba86482f890e1c386dc22594 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:34.080717 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Authenticating user token {{(pid=81875) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:46:34.082181 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.fernet_utils [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81875) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:46:34.096454 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656a74-cc14-4ffd-9c0a-34aeb34d4f3d admin admin] Validating token access rules against request {{(pid=81875) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:46:34.157789 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fd5eb0c7a1f4bdbb078438d56380575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb1de965aba54ca98b770d7165c4b67e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81875) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:46:34.158424 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:46:34.158708 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:46:34.158985 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] PATH_INFO: `/v3/projects/7644cad195174da080a9989cec76033d` {{(pid=81875) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:46:34.160148 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7644cad195174da080a9989cec76033d)` {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:46:34.162050 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] RBAC: Authorization granted {{(pid=81875) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:46:34.170483 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:34.266703 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7644cad195174da080a9989cec76033d', 'request_id': 'req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:34.273519 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7644cad195174da080a9989cec76033d', 'request_id': 'req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:34.398718 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7644cad195174da080a9989cec76033d', 'request_id': 'req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95'} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:34.409869 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] The token cache is being invalidate because project 7644cad195174da080a9989cec76033d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81875) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 21:46:34.410158 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:34.410853 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG keystone.notifications [None req-bc3c4ae5-96ee-4ad1-9e3e-a3151b1dab95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81875) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 21:46:34.412422 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: [pid: 81875|app: 0|req: 1470/2948] 10.222.0.65 () {58 vars in 1233 bytes} [Mon Aug 29 21:46:34 2022] DELETE /identity/v3/projects/7644cad195174da080a9989cec76033d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 21:46:43.804083 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=57,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:46:44.406831 dv-cinder-854264-2-iscsi devstack@keystone.service[81875]: DEBUG dbcounter [-] [81875] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81875) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 21:49:53.511848 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Authenticating user token {{(pid=81874) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 21:49:53.513406 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:49:53.589389 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b766c0-a274-4593-a07a-55f3268d4541 admin admin] Validating token access rules against request {{(pid=81874) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 21:49:53.650125 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c39ceb5a1fb48328b9ca050172da36e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ffba55e45324a748e9fcb000ff276ec', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81874) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 21:49:53.650737 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] REQUEST_METHOD: `GET` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 21:49:53.651017 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] SCRIPT_NAME: `/identity` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 21:49:53.651290 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81874) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 21:49:53.653189 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.fernet_utils [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81874) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 21:49:53.664521 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 21:49:53.666108 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc44cd8-62f3-4b56-8b06-7525198b5d9a service placement] RBAC: Authorization granted {{(pid=81874) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 21:49:53.676522 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: [pid: 81874|app: 0|req: 1479/2949] 10.222.0.65 () {60 vars in 1365 bytes} [Mon Aug 29 21:49:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 21:50:03.671874 dv-cinder-854264-2-iscsi devstack@keystone.service[81874]: DEBUG dbcounter [-] [81874] Writing DB stats keystone:SELECT=58 {{(pid=81874) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}